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EXECUTIVE  SUMMARY 

The  LCNA  Methodology  Report  provides  detailed  information  about  the  data  collection  and  analysis 
methods  used  for  the  Special  Operations  Forces  (SOF)  Language  and  Culture  Needs  Assessment  (LCNA) 
Project.  This  project  combined  focus  groups  and  a  web-based  survey  across  the  SOF  community  to 
capture  the  perspectives  of  SOF  operators,  leaders  and  support  personnel  on  language  and  culture 
requirements,  issues,  policy  and  resources.  Focus  groups  and  surveys  are  often  used  together  as  an 
organizational  needs  assessment  technique,  and  the  practice  of  integrating  multiple  data  collection 
techniques  is  known  to  improve  the  quality  of  findings.  Given  the  constraints  (e.g.,  operations  tempo)  and 
complexity  of  data  collection  (e.g.,  collection  across  multiple  services  and  organizations)  in  the  SOF 
community,  the  methods  used  in  this  study  were  selected  to  collect  the  most  comprehensive  data  on 
language  and  culture  in  the  SOF  community.  This  report  provides  information  so  readers  can  judge  the 
rigor  and  appropriateness  of  the  study  methodology  and  its  execution,  allowing  them  to  determine  the 
level  of  confidence  they  have  in  the  findings  presented  in  the  issue  reports.  This  report  provides  specific 
details  of  the  SOF  LCNA  focus  groups,  such  as  the  processes  followed  for  conducting  the  focus  groups 
and  for  coding  and  analyzing  the  focus  group  data.  A  description  of  the  SOF  LCNA  survey  methodology 
from  survey  development,  deployment  and  administration  to  the  steps  used  in  cleaning  and  analyzing  the 
data  are  presented.  This  report  provides  no  information  on  participation  in  the  project.  That  information  is 
provided  in  the  LCNA  Participation  Report  (Technical  Report  # 2010011003 ).  The  Participation  and 
Methodology  Reports  form  foundation  reports  for  subsequent  issue  reports.  By  including  the  majority  of 
the  methodology  and  participation  information  in  these  reports,  redundancy  is  eliminated  from  all  the 
issue  reports,  allowing  the  focus  to  be  on  aspects  of  the  methodology  and  participation  unique  to  the 
specific  issue  report. 

The  Special  Operations  Forces  Culture  and  Language  Office  (SOFCLO)  commissioned  the  SOF 
Language  and  Culture  Needs  Assessment  Project  (LCNA)  to  gain  insights  on  language  and  culture 
capability  and  issues  across  the  United  States  Special  Operations  Command  (USSOCOM).  The  goal  of 
this  organizational-level  needs  assessment  is  to  inform  strategy  and  policy  to  ensure  SOF  personnel  have 
the  language  and  culture  skills  needed  to  conduct  their  missions  effectively.  Data  were  collected  between 
March  and  November,  2009  from  personnel  in  the  SOF  community,  including  operators  and  leaders. 
Findings,  gathered  via  focus  groups  and  a  web-based  survey,  will  be  presented  in  a  series  of  reports 
divided  into  three  tiers.  The  specific  reports  in  each  of  these  tierss  will  be  determined  and  contracted  by 
the  SOFCLO.  As  originally  planned,  Tier  I  Reports  focus  on  specific,  limited  issues  [e.g.,  Inside/Outside 
Area  of  Operations  (AOR)  Use  of  Cultural  Knowledge,  Inside  AOR  Use  of  Language]  Tier  II  Reports 
will  integrate  and  present  the  most  important  findings  across  related  Tier  I  reports  (e.g.,  Use  of  Language 
and  Culture  on  Deployment)  while  including  additional  data  and  analysis  on  the  topic.  One  Tier  III 
Report  will  present  the  most  important  findings,  implications,  and  recommendations  across  all  topics 
explored  in  this  project.  The  remaining  Tier  III  reports  will  present  findings  for  specific  SOF 
organizations  [e.g.,  Air  Force  Special  Operations  Command  (AFSOC),  Special  Forces  (SF)  Command]. 
Two  foundational  reports  document  the  methodology  and  participants  associated  with  this  project.  As 
mentioned,  the  additional  reports  will  be  determined  by  the  SOFCLO  and  may  differ  from  what  was 
originally  planned. 

The  LCNA  Methodology  Report  is  a  foundational  report  for  the  LCNA  project  that  is  cited  by  all  other 
Tier  I,  Tier  II,  and  Tier  III  Reports.  Other  reports  will  reference  this  foundational  report  in  order  to  reduce 
the  length  of  subsequent  reports  and  provide  a  single  source  for  information  about  the  procedures, 
materials,  and  analyses  used  in  this  project.  The  final  reports  produced  will  be  determined  by  the 
SOFCLO. 
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SECTION  I:  REPORT  OVERVIEW 
LCNA  Methodology  Report  Purpose 

The  LCNA  Methodology  Report  provides  detailed  information  about  the  data  collection  and  analysis 
methods  used  for  the  Special  Operations  Forces  (SOF)  Language  and  Culture  Needs  Assessment  (LCNA) 
Project.  This  project  combined  focus  groups  and  a  web-based  survey  across  the  SOF  community  to 
capture  the  perspectives  of  SOF  operators,  leaders  and  support  personnel  on  language  and  culture 
requirements,  issues,  policy  and  resources.  Focus  groups  and  surveys  are  often  used  together  as  an 
organizational  needs  assessment  technique,  and  the  practice  of  integrating  multiple  data  collection 
techniques  is  known  to  improve  the  quality  of  findings.  Given  the  constraints  (e.g.,  operations  tempo)  and 
complexity  of  data  collection  (e.g.,  collection  across  multiple  services  and  organizations)  in  the  SOF 
community,  the  methods  used  in  this  study  were  selected  to  collect  the  most  comprehensive  data  on 
language  and  culture  in  the  SOF  community.  This  report  provides  information  so  readers  can  judge  the 
rigor  and  appropriateness  of  the  study  methodology  and  its  execution,  allowing  them  to  determine  the 
level  of  confidence  they  have  in  the  findings  presented  in  the  issue  reports.  This  report  provides  specific 
details  of  the  SOF  LCNA  focus  groups,  such  as  the  processes  followed  for  conducting  the  focus  groups 
and  for  coding  and  analyzing  the  focus  group  data.  A  description  of  the  SOF  LCNA  survey  methodology 
from  survey  development,  deployment  and  administration  to  the  steps  used  in  cleaning  and  analyzing  the 
data  are  presented. 

Focus  groups  were  conducted  between  2  March  and  11  June  2009.  Section  11:  Focus  Group  Protocol  and 
Logistics  provides  an  overview  of  focus  group  roles  and  the  processes  followed  for  the  SOF  LCNA  focus 
groups.  Section  111:  Focus  Group  Analysis  details  the  data  analysis  involved  in  coding  the  focus  group 
data.  The  web-based  survey  was  conducted  between  26  October  and  24  November  2009.  A  description  of 
the  SOF  LCNA  survey  development  and  administration  is  provided  in  Section  IV:  Survey  Protocol  and 
Logistics.  Section  V:  Survey  Analysis  provides  steps  used  in  cleaning  the  data,  analyzing  open-ended 
comments,  and  quantitative  items. 

Project  Purpose 

The  Special  Operations  Forces  Culture  and  Language  Office  (SOFCLO)  commissioned  the  SOF 
Language  and  Culture  Needs  Assessment  Project  (LCNA)  to  gain  insights  on  language  and  culture 
capability  and  issues  across  the  United  States  Special  Operations  Command  (USSOCOM).  The  goal  of 
this  organizational-level  needs  assessment  is  to  inform  strategy  and  policy  to  ensure  SOF  personnel  have 
the  language  and  culture  skills  needed  to  conduct  their  missions  effectively.  Data  were  collected  between 
March  and  November,  2009  from  personnel  in  the  SOF  community,  including  operators  and  leaders. 
Findings,  gathered  via  focus  groups  and  a  web-based  survey,  will  be  presented  in  a  series  of  reports 
divided  into  three  tiers.  The  specific  reports  in  each  of  these  tiers  will  be  determined  and  contracted  by  the 
SOFCLO.  As  originally  planned,  Tier  1  Reports  focus  on  specific,  limited  issues  [e.g.,  Inside/Outside 
Area  of  Operations  (AOR)  Use  of  Cultural  Knowledge,  Inside  AOR  Use  of  Language]  Tier  II  Reports 
will  integrate  and  present  the  most  important  findings  across  related  Tier  I  reports  (e.g.,  Use  of  Language 
and  Culture  on  Deployment)  while  including  additional  data  and  analysis  on  the  topic.  One  Tier  III 
Report  will  present  the  most  important  findings,  implications,  and  recommendations  across  all  topics 
explored  in  this  project.  The  remaining  Tier  III  reports  will  present  findings  for  specific  SOF 
organizations  [e.g.,  Air  Force  Special  Operations  Command  (AFSOC),  Special  Forces  (SF)  Command]. 
Two  foundational  reports  document  the  methodology  and  participants  associated  with  this  project.  As 
mentioned,  the  additional  reports  will  be  determined  by  the  SOFCLO  and  may  differ  from  what  was 
originally  planned. 
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See  Appendix  A  for  more  details  about  the  2009  SOF  LCNA  Project  and  initially  planned  report 
structure. 

Relationship  of  LCNA  Methodology  Report  to  the  LCNA  Project 

The  LCNA  Methodology  Report  is  a  foundational  report  for  the  LCNA  project  that  is  cited  by  all  other 
Tier  I,  Tier  II,  and  Tier  III  Reports.  Other  reports  will  reference  this  foundational  report  in  order  to  reduce 
the  length  of  subsequent  reports  and  provide  a  single  source  for  information  about  the  procedures, 
materials,  and  analyses  used  in  this  project.  The  final  reports  produced  will  be  determined  by  the 
SOFCLO. 
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SECTION  II:  FOCUS  GROUP  PROTOCOL  AND  LOGISTICS 

Consistent  with  the  SOF  Language  Transformation  Strategy  Needs  Assessment  Project  conducted  in 
2005,  the  first  phase  of  data  collection  for  the  SOF  LCNA  study  consisted  of  focus  groups.  A  focus  group 
is  a  qualitative  research  method  used  to  collect  responses  from  a  group  of  people  familiar  with  a  topic, 
service,  experience,  or  product  (Zemke  &  Kramlinger,  1982).  Typically,  focus  groups  are  composed  of  6 
to  12  individuals  who  have  been  selected  for  participation  from  a  specified  population  based  on  a  pre- 
established  set  of  criteria.  Focus  group  participants  are  typically  selected  for  their  perspective  on  the  focal 
issue(s)  being  assessed  and  are  often  subject  matter  experts  on  the  topics  to  be  discussed.  A  trained 
facilitator  leads  the  participants  through  a  pre-determined  series  of  questions,  discussions,  or  exercises. 
The  facilitator  may  probe  interesting  responses  further.  The  questions  and  exercises  used  in  the  focus 
groups  were  adapted  from  the  2005  focus  group  protocols  to  reflect  the  purpose  of  the  current  project. 

In  this  study,  30  focus  groups  were  planned  at  units  across  USSOCOM  (see  Table  1).  SOFCLO  asked  the 
units  to  provide  participants  who  were  SOF  operators  with  varying  degrees  of  deployment  and  language 
experience.  Due  to  scheduling  constraints  and  the  lack  of  availability  of  operators  at  units  because  of 
deployments,  only  23  focus  groups  were  conducted. 

Table  1.  SOF  LCNA  Focus  Groups 


Unit 

Location 

#  Focus  Groups 
Planned 

#  Focus  Groups 
Conducted 

AFSOC 

AFSOC 

Hurlburt,  FL 

3 

2 

USASOC 

95th  CA  Bde 

Ft.  Bragg,  NC 

2 

2 

4th  POG 

Ft.  Bragg,  NC 

2 

2 

1st  SFG(A) 

Ft.  Lewis,  WA 

2 

2 

1st  BN  1st  SFG(A) 

Okinawa,  Japan 

1 

1 

5th  SFG(A) 

Ft.  Campbell,  KY 

1 

1 

10th  SFG(A) 

Ft.  Carson,  CO 

1 

1 

1st  BN  10th  SFG(A) 

Stuttgart,  Germany 

2 

2 

19th  SFG(A) 

Draper,  UT 

2 

1 

3rd  SFG  (A) 

Ft.  Bragg,  NC 

1 

- 

7th  SFG  (A) 

Ft.  Bragg,  NC 

1 

- 

20th  SFG  (A) 

Birmingham,  AL 

1 

- 

20th  SFG  (A) 

Springfield,  MA 

1 

- 

WARCOM 

NSWU-2 

Stuttgart,  Germany 

1 

1 

NSWU-3 

Bahrain 

1 

1 

NAVSCIATTS  &  Special  Boat  Team 

22 

GulfPort,  MS 

1 

1 

NSWG-1 

Coronado,  CA 

1 

1 

NSWG-2 

Little  Creek,  VA 

1 

1 

NSWG-4 

Little  Creek,  VA 

1 

1 

NSWC 

Coronado,  CA 

1 

1 

MARSOC 

MARSOC  (East) 

Camp  Lejeune,  NC 

2 

1 

MARSOC  (West) 

Camp  Pendleton,  CA 

1 

1 
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Focus  Group  Roles 

Moderator/Facilitator 

For  a  focus  group  to  be  constructive,  it  must  be  moderated  by  an  experienced  professional  (McClelland, 
1994;  Zemke  &  Kramlinger,  1982).  The  facilitator  focuses  participant  discussion  skillfully  to  elicit  and 
gather  information  that  addresses  the  objectives  of  the  study.  The  facilitator  guides  the  discussion, 
encourages  the  participation  of  all  group  members,  keeps  the  discussion  on  track,  and  probes  interesting 
responses  further.  The  projects’  lead  researchers  moderated  the  SOF  LCNA  focus  groups  and  followed 
focus  group  moderation  best  practices  as  described  in  Focus  Group  Protocol  and  Logistics  (Technical 
Report#  2009011003). 

Scribe 

To  capture  information  effectively,  thematic  notes  should  be  taken  by  a  scribe  for  each  focus  group. 
Additionally,  the  session  should  be  recorded  and  transcribed  for  analysis.  The  thematic  notes  are  vital 
because  the  scribe  can  capture  information  that  the  transcript  will  not  reveal.  Importantly,  the  scribe’s 
data  are  available  immediately  to  adjust  focus  group  protocol  or  questions.  Plus,  if  the  recording  is 
unclear  or  damaged,  there  is  a  thematic  and  contextual  record  of  the  group. 

Each  SOF  LCNA  focus  group  used  a  scribe  in  addition  to  the  sessions  being  recorded  and  transcribed. 
The  scribe’s  main  responsibility  was  to  take  detailed  notes  of  the  discussion.  The  scribes  used  Livescribe 
Smart  Pens  to  take  notes  during  the  focus  group.  The  Livescribe  Smart  Pens  captured  both  written  and 
audio  notes  and  were  used  as  an  additional  form  of  backup.  The  scribes  also  noted  any  questions  or 
procedural  changes  for  future  focus  groups  and  discussed  them  with  the  moderator.  See  Focus  Group 
Protocol  and  Logistics  (Technical  Report #  2009011003)  for  more  details  on  scribe  best  practices. 

SOFCLO  Representative 

A  third  role  was  added  for  the  SOF  LCNA  focus  groups;  the  SOFCLO  representative  provided  legitimate 
authority  and  served  as  the  event  sponsor.  Once  the  focus  groups  started,  the  SOFCLO  representative 
welcomed  the  participants  and  described  the  project.  Then,  the  representative  turned  the  group  over  to  the 
moderator/facilitator.  The  SOFCLO  representative  also  served  as  secondary  scribe  when  there  were  three 
personnel  to  run  the  group,  and  the  primary  scribe  when  there  were  two  team  members. 

Focus  Group  Process 

Planning  Focus  Groups 

The  SOFCLO  representative  made  initial  contact  with  units  to  make  arrangements  for  the  focus  groups. 
The  moderator  and  scribe  were  responsible  for  providing  the  necessary  materials  for  the  focus  groups  and 
for  conducting  the  focus  groups. 

Protocol 

The  focus  group  protocol  used  in  the  current  study  was  very  similar  to  the  one  used  in  the  previous  study, 
allowing  for  meaningful  comparison  in  some  areas.  However,  the  current  protocol  was  modified  to  meet 
the  objectives  of  the  current  study  as  well  as  to  incorporate  lessons  learned.  For  example,  the  current 
study  included  questions  related  to  the  use  of  culture  on  missions.  Culture  was  not  a  focus  of  the  pervious 
study. 

In  the  current  study,  participants  received  a  copy  of  the  ground  rules  which  focused  on  not  sharing 
confidential  information,  providing  only  first  hand  experiences,  and  respecting  each  other’s  view  (see 
Appendix  B  for  the  complete  list  of  ground  rules).  Participant  surveys  were  distributed  to  gather 
information  about  each  participant’s  SOF  component,  military  occupational  specialty  (MOS),  and 
language  background.  The  survey  also  gave  the  participants  an  opportunity  to  write  additional  comments 
for  each  portion  of  the  focus  group  (see  Appendix  C).  Participants  were  assigned  a  participant  number  at 


2/25/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011002] 


Page  7 


SOF  Language  and  Culture  Needs  Assessment  Project 


Methodology  Report 


the  beginning  of  each  focus  group.  This  number  was  used  during  the  discussion  to  protect  participant 
confidentiality. 

The  full  protocol  for  SOF  LCNA  focus  groups  consisted  of  three  blocks  designed  to  elicit  information 
related  to  major  areas  of  interest.  The  focus  group  questions  and  activity  were  similar  to  the  protocol  used 
in  the  previous  study,  but  topics  and  order  were  modified  somewhat  to  reflect  current  priorities.  The  first 
block  contained  questions  about  language  usage  and  capability  requirements  as  well  as  cultural 
knowledge  and  awareness  requirements.  The  second  block  asked  participants  to  provide  their  views  on 
language  and  culture  training,  barriers  to  language  and  culture  skills,  motivators  for  language  and  culture 
training,  and  language  testing.  The  third  block  was  a  participant  activity  for  developing  a  language  and 
culture  training  strategy.  Each  block  lasted  approximately  one  hour,  with  short  breaks  in  between.  An 
abbreviated  protocol  was  developed  to  accommodate  focus  groups  in  which  time  was  limited.  In  some 
instances,  the  full  protocol  was  followed  for  block  1  and  block  2;  however  time  necessitated  the 
elimination  of  the  block  3  activity.  The  moderator  followed  the  interview  guide,  but  deviated  as  necessary 
to  capture  information.  The  full  and  abbreviated  protocols  are  provided  in  Appendices  D  and  E 
respectively. 

Note  Formats 

Scribes  followed  a  standardized  format  for  taking  notes.  On  each  notes  page,  the  question  was  captured 
along  with  the  participant’s  number  and  the  general  theme  of  the  participant’s  comment.  At  the 
conclusion  of  the  focus  group,  the  scribe  reviewed  the  notes  and  added  any  details  remembered,  but  did 
not  have  time  to  write  out  fully.  A  back-up  copy  of  the  recordings  was  also  uploaded  from  the  Livescribe 
Smart  Pens  to  an  encrypted  flash  drive. 

Session  Recordings 

Audio  recordings  were  captured  using  a  Livescribe  Smart  Pen  and  two  digital  audio  recorders.  The  two 
recorders  were  used  in  the  event  that  one  of  the  recorders  malfunctioned  (which  happened  in  the  previous 
study).  The  recorders  were  not  turned  on  while  participants  introduced  themselves  to  the  group  or  while 
participants  were  planning  their  activity.  Participants  were  notified  each  time  the  recorders  were  turned 
on. 

Transcription 

Data  from  focus  groups  can  be  captured  in  various  forms.  Transcript  based  data  is  the  most  complete  form 
and  is  best  used  for  complex  studies.  This  approach  relies  on  notes  taken  by  a  scribe  and  a  verbatim 
transcript  of  an  audio  recording  from  the  focus  groups  (Krueger  &  Casey,  2009).  Due  to  the  complexity 
of  the  SOF  LCNA  project,  a  transcript  based  approach  was  used  and  was  supplemented  by  scribe  notes. 

After  the  focus  groups  were  completed,  a  firm  with  expertise  transcribed  the  sessions.  Transcribers 
received  the  audio  recordings  from  both  recording  devices  used  in  the  focus  group  sessions.  Each  focus 
group  transcription  was  returned  as  an  individual  word  document.  Transcribers  followed  these  guidelines: 
o  Any  time  the  moderator  speaks  should  be  designated  with  a  “Q”  at  the  beginning, 
o  Any  time  a  focus  group  participant  speaks  should  be  designated  with  a  “P”  and  their  specific 
participant  number  (i.e.,  P3  for  participant  number  3).  Participant  numbers  should  be  indicated  if 
and  only  if  it  is  abundantly  clear  which  participant  was  speaking.  If  it  is  unclear,  it  should  be 
designated  as  “P?”. 

o  If  words/acronyms  are  unclear,  use  consistent  phonetic  spelling  throughout  the  transcript, 
o  Listen  to  the  second  recorder  in  attempt  to  fill-in  any  unclear/inaudible  sections  from  the  first 
recorder.  If  sections  are  still  unclear/inaudible,  indicate  “[indiscernible]”  in  these  sections. 
Transcriptions  were  returned  as  they  were  completed  via  an  encrypted  flashdrive.  The  transcripts  were 
then  stored  and  backed  up  on  an  encrypted  flash  drive  and  stored  in  a  fireproof  safe. 
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SWA  reviewed  each  transcript  for  accuracy  and  completeness.  Missing  information  was  filled  in  when  it 
was  clearly  discernable  from  the  two  audio  recordings.  In  addition,  notes  and  recordings  from  the 
LiveScribe  Smart  Pens  were  used  to  ensure  that  the  final  transcripts  were  as  complete  as  possible.  For 
instance,  during  the  focus  groups  scribes  recorded  the  participant  number  during  the  discussion.  Thus, 
using  these  notes,  missing  participant  numbers  were  added  to  the  transcripts.  If  clearly  identifiable,  voice 
recognition  was  used  to  identify  participants. 

During  the  review  process,  LCNA  Focus  Group  Summary  Documents  were  completed  to  provide  a 
condensed  summary  of  relevant  group  attributes  (e.g.,  SOF  component),  main  ideas  or  themes  expressed 
by  the  focus  groups,  and  recurring  issues  identified  by  participants.  These  summaries  served  as  initial 
steps  in  the  survey  development  process,  described  in  Section  IV  of  this  report. 
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SECTION  III:  FOCUS  GROUP  ANALYSIS 

The  transcripts  provided  a  complete  record  of  the  focus  groups;  however  data  analysis  was  needed  to 
identify  themes  from  the  focus  groups.  Data  analysis  began  with  coding  the  transcripts  with  attribute, 
structural,  and  content  codes.  To  ensure  the  quality  of  the  analysis,  multiple  coders  were  used  and 
intercoder  agreement  was  calculated.  The  prevalence  of  themes  was  then  identified  with  frequency 
counts. 

Focus  Group  Transcript  Analysis 

The  transcripts  were  imported  into  MAXQDA,  a  qualitative  analysis  program  that  provides  a  structure  for 
organizing  and  assigning  codes  to  segments  of  the  discussion,  while  still  maintaining  the  integrity  of  the 
data  in  its  original  form.  All  researchers  involved  in  the  focus  group  analysis  were  trained  on  the  use  of 
MAXQDA. 

The  data  from  the  focus  groups  were  coded  in  multiple  phases:  attribute  coding,  structural  coding,  and 
content  coding.  Each  phase  provided  a  more  detailed  analysis  of  the  data. 

Attribute  Coding 

Attribute  coding  provides  “essential  information  about  the  data  and  demographic  characteristics  of  the 
participants  for  future  management  and  reference”  (Saldana,  2009,  p.  55).  Relevant  information  about 
participants,  location,  and  other  related  components  of  the  study  were  coded  as  focus  group  attributes.  For 
this  study,  the  following  attributes  were  coded:  SOF  component,  unit,  number  of  participants,  location, 
type  of  protocol  (full  or  abbreviated),  and  inclusion/exclusion  of  the  block  3  activity.  These  data  were 
recorded  at  the  time  of  focus  group  administration,  entered  into  MAXQDA,  and  linked  to  the  respective 
transcript.  While  analyzing  the  focus  group  data,  coders  were  able  to  reference  this  information. 
Additionally,  within  each  transcript,  codes  were  assigned  to  differentiate  between  moderator  questions 
and  participant  responses.  The  block  3  activity  portions  of  the  transcript  were  also  coded  to  reflect  which 
activity  (i.e.,  option  1  or  2)  was  selected  by  the  participants. 

Structural  Coding 

Structural  coding  works  best  for  data  collected  through  discrete  questions  and  probes  (e.g.,  focus  groups) 
that  are  repeated  across  multiple  files  (i.e.,  groups)  in  a  data  set  (MacQueen,  McFellan-Femal,  Bartholow, 
&  Milstein,  2008;  Saldana,  2009).  Structural  coding  makes  “subsequent  analyses  easier  by  identifying  all 
of  the  text  associated  with  a  particular  question  and  its  associated  probes”  (MacQueen  et  al.,  2008,  p. 

124).  This  step  is  essential  for  grouping  similarly  themed  discussion  to  prepare  for  more  detailed  content 
coding. 

Researchers  developed  structural  codes  using  the  focus  group  protocol: 

•  Mission  example  language 

•  Mission  example  culture 

•  Core  SOF  tasks/modalities 

•  Fanguage  capability  on  the  team 

•  Use  of  job  aids 

•  Placement  of  language/culture  training 

•  Effective  experiences  in  language/culture  training 

•  Ineffective  experiences  in  language/culture  training 

•  Barriers,  motivators  (other  than  pay) 

•  Monetary  incentives  (Foreign  Fanguage  Proficiency  Pay/Foreign  Fanguage  Proficiency  Bonus) 
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•  Language  testing 

•  Integration  with  other  training 

•  Distance  Learning/Technology-delivered  Training 

•  Special  Operation  Forces  Tele-training  System 

•  Suggestions 

•  Block  3  activity  1 

•  Block  3  activity  2 

The  data  were  broken  into  broad,  similarly-themed  segments  for  this  phase  of  structural  coding.  Within 
each  segment,  the  question  from  the  moderator  and  the  complete  response  from  the  participant(s)  were 
included.  Additionally,  any  dialog  between  the  moderator  and  the  participant  resulting  from  the  initial 
question  was  captured  in  the  segment.  Including  the  full  elicitation  preserved  both  the  flow  of  the  focus 
group  discussion  and  the  full  context  of  the  dialog  on  a  particular  topic  (MacQueen  et  ah,  2008).  Coders 
read  the  transcripts  and  placed  the  selected  text  segments  into  the  code  designated  as  best  describing  the 
content  of  the  segment.  The  structural  codes  were  used  to  classify  the  data  for  inclusion  in  the  SOF 
LCNA  Project  Database  and  were  also  used  as  the  basis  for  organizing  subsequent  analysis. 

Codebook  Development 

Codebooks  and  coding  instructions  are  developed  as  part  of  the  process  for  creating  a  shared  mental 
model  among  coders  for  applying  the  codes.  By  establishing  a  detailed  guideline  for  applying  content 
codes,  the  interchangeability  of  coders  is  enhanced  (i.e.,  a  new  coder  could  be  added  to  the  process  at  any 
time  without  altering  the  results  of  the  study).  Furthermore,  coding  instructions  allow  others  to  replicate 
the  results  of  the  study  or  to  apply  the  same  codes  to  additional  information  collected  at  a  later  date. 

For  this  project,  in  order  to  establish  relevant  content  codes,  text  segments  assigned  to  each  structural 
code  were  reviewed  for  recurring  themes.  Research  team  members  reviewed  themes  to  ensure  they  were 
represented  and  to  further  clarify  any  unclear  codes.  Then  the  coders  developed  definitions  for  each  theme 
and  rules  about  when  the  codes  should  and  should  not  be  applied  in  accordance  with  established 
guidelines  (MacQueen,  McLellan,  Kay,  &  Milstein,  1998).  Research  team  members  conducted  a  final 
review  of  the  codes  and  their  definitions  for  clarity  and  conciseness.  Where  appropriate,  the  relevant 
codebook  will  be  attached  to  its  corresponding  report. 

Unitizing 

Unitizing  is  the  process  of  creating  distinct  units  of  text  within  the  data  set.  This  process  helps  determine 
which  portions  of  text  to  include  in  further  analysis  and  which  portions  are  not  relevant  to  the  research 
purpose  (Krippendorff,  2004).  In  addition,  this  process  allows  researchers  to  further  segment  elements  of 
the  discussion  within  the  structural  codes  and  eases  the  coding  burden  in  subsequent  analysis.  There  are 
several  different  methods  for  unitizing  data,  but  in  general  it  is  recommended  that  each  unit  be  separate 
and  distinct  from  all  other  units  and  that  units  should  be  as  small  as  possible  while  still  being  meaningful. 

Before  the  content  codes  were  applied,  coders  unitized  the  data.  Researchers  followed  specific  guidelines 
listed  below: 

1.  Each  discrete  unit  should  represent  one  theme/critical  incident. 

2.  A  discrete  unit  should  be  understood  on  its  own  without  additional  context. 

3.  A  discrete  unit  may  encompass  multiple  exchanges. 

4.  A  discrete  unit  should  NOT  reflect  multiple  participants  who  state  the  same  point  of  view. 

5.  A  discrete  unit  should  NOT  include  moderator  questions  when  the  participant’s  response  can  be 
understood  alone. 
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Content  Coding 

Once  the  data  are  unitized,  content  codes  can  be  applied.  This  approach  further  identifies  more  specific 
themes  that  emerge  from  participants’  answers  to  the  focus  group  prompts.  Each  discrete  unit  of  text 
identified  during  unitizing  is  content  coded.  Although  it  is  preferable  if  a  single  code  can  be  applied  to  a 
single  unit  of  text,  simultaneous  coding  (i.e.,  applying  multiple  codes  to  a  single  unit  of  text)  is  often 
necessary  with  complex  topics  and  discussion.  This  form  of  coding  can  be  used  when  it  is  impossible  to 
capture  the  sentiment  of  the  text  with  a  single  code  (Saldana,  2009). 

Coders  began  the  content  coding  process  by  reviewing  the  relevant  codebooks  and  making  any  necessary 
modifications.  The  unitized  segments  were  then  independently  coded.  Within  the  content  coding  phase, 
simultaneous  coding  techniques  were  used  to  accommodate  the  need  to  describe  specific  units  using 
multiple  characteristics. 

Quality  Assurance 

Coder  training,  the  use  of  multiple  coders,  and  the  calculation  of  intercoder  agreement  are  used  to 
enhance  the  quality  of  analysis  throughout  the  coding  process. 

Prior  to  each  phase  of  coding,  each  coder  was  trained  to  ensure  that  each  coder  understood  and  executed 
the  same  process.  Training  sessions  included  a  review  of  the  purpose  of  each  phase  of  the  coding  process 
along  with  guidelines  for  coding,  practice  coding,  and  calculating  intercoder  agreement  to  determine  if  the 
coders  were  ready  to  proceed.  Two  coders  were  assigned  to  each  topic  area  for  structural  coding, 
unitizing,  and  content  coding.  Coders  worked  independently  to  assign  codes  to  segments  of  text  and  then 
met  to  review  disagreements  and  discuss  to  consensus. 

Researchers  considered  several  methods  of  calculating  agreement  for  this  project.  The  most  appropriate 
index  for  agreement  was  Krippendorff’ s  alpha,  a  reliability  measure  that  accommodates  nominal  data  and 
agreement  due  to  chance.  However,  Krippendorff  s  alpha  does  not  accommodate  the  unitization  process 
or  simultaneous  coding.  Due  to  these  restrictions,  Guetzkow’s  U  and  absolute  agreement  were  used  for 
this  project. 

During  unitization,  Guetzkow’s  U  (Guetzkow,  1950)  was  most  appropriate  for  assessing  intercoder 
agreement.  Agreement  was  calculated  on  an  item  by  item  basis  for  each  content  area  in  the  unitization 
process.  The  coders  discussed  any  text  segments  where  they  disagreed  and  reached  consensus  about  the 
appropriate  unit  to  apply. 

Absolute  agreement  was  appropriate  for  structural  coding  and  content  coding.  This  method  assessed 
intercoder  agreement  by  calculating  the  percentage  of  units  that  the  coders  agreed  upon  (Bernard  &  Ryan, 
2010).  Higher  levels  of  absolute  agreement  were  an  indication  that  the  coders  were  applying  the  codes 
consistently. 

Agreement  was  calculated  at  multiple  points  during  coding  for  each  content  area.  The  codebooks  were 
reviewed  and  revised  when  the  agreement  was  low  and  indicated  that  the  coders  had  consistent 
differences  in  the  use  of  the  codes.  When  agreement  standards  were  not  met  and  in  all  cases  of 
disagreement,  the  coders  discussed  to  agreement. 

Focus  Group  Quantitative  Analysis 

Following  the  qualitative  analysis,  the  frequency  of  text  segments  assigned  to  each  code  was  calculated  in 
MAXQDA  to  assess  the  prevalence  of  themes  across  all  focus  groups.  MAXQDA  also  allowed  for 
complex  retrieval  of  segments,  which  produced  the  frequency  for  these  segments.  Results  from  analyses 
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of  the  focus  group  data  were  used  to  supplement  the  results  of  the  quantitative  survey  analyses  and  are 
presented  in  related  project  reports. 
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SECTION  IV:  SURVEY  PROTOCOL  AND  LOGISTICS 

The  second  phase  of  data  collection  included  a  web-based  survey.  Web-based  surveys  have  been  used  as 
effective  data  collection  tools  in  the  SOF  community  (e.g.,  The  SOF  Language  Transformation  Strategy 
Needs  Assessment  Project  Technical  Report  #20040606',  Thompson,  Surface,  Martin,  &  Sanders,  2003; 
Thompson  &  Surface,  2009).  Surveys  are  often  used  in  combination  with  focus  group  information.  This 
combination  allows  researchers  to  capitalize  on  the  unique  contributions  and  strengths  of  each  data 
collection  method.  Focus  groups  provide  greater  depth  of  information  on  selected  topics  while  surveys 
provide  a  wider  spectrum  of  information  by  targeting  a  larger  number  of  participants.  Additionally,  focus 
groups  can  inform  the  development  of  a  survey.  Common  themes  that  emerge  from  focus  group  results 
inform  survey  topics  and  appropriate  expressions/language  familiar  to  the  target  audience  (Morgan, 

1996). 

Survey  Development 

Item  Planning 

Survey  items  should  reflect  the  purpose  of  the  project  and  should  be  grounded  in  the  project’s  research 
questions.  Therefore,  The  SOF  Language  Transformation  Strategy  Needs  Assessment  Project  served  as  an 
initial  guide  for  the  research  questions  for  this  project  ( Technical  Report  #20040606).  Additional  topic 
areas  of  interest  and  research  questions  were  identified  based  on  current  client  interests  (i.e.,  assessing 
culture  needs)  and  major  themes  identified  in  the  23  focus  groups  conducted  with  SOF  personnel.  One  of 
the  advantages  of  conducting  focus  groups  prior  to  developing  the  survey  is  the  ability  to  incorporate 
current  issues  from  the  field  for  validation  with  a  larger  and  more  representative  group  of  participants. 

Item  Writing 

Best  practices  indicate  that  multiple  sources  should  be  consulted  when  writing  items.  For  instance, 
information  gathered  during  focus  groups  can  be  used  to  generate  and  refine  survey  topics  and  items 
(Morgan,  1996).  Relevant  literature  may  also  be  consulted  during  item  generation  (Hinkin,  1998).  The  use 
of  focus  groups  and  existing  literature  provides  confidence  that  the  content  of  the  survey  will  be  relevant 
to  the  project’s  purpose. 

Consistent  with  best  practices,  three  sources  were  consulted  when  generating  items  for  the  SOF  LCNA 
survey.  Item  development  began  with  a  review  of  The  SOF  Language  Transformation  Strategy  Needs 
Assessment  Project  survey  items  ( Technical  Report  #20040606).  The  items  were  reviewed  for  their 
relevance  to  the  present  project  and  to  determine  if  items  needed  to  be  refined.  An  initial  review  of  the 
focus  groups  and  literature  relevant  to  the  project  purpose  were  also  used  to  inform  item  development. 
Information  from  each  of  these  areas  was  used  to  draft  items  in  accordance  with  the  project  purpose. 

In  order  to  write  items  that  yield  useful  results,  it  is  essential  to  follow  guidelines  such  as  the  following 
which  were  used  in  the  present  project  (adapted  from  Fowler,  2002). 

1.  Write  short,  simple  statements. 

2.  Use  language  that  is  familiar  to  the  target  audience. 

3.  Use  a  consistent  perspective  -  don’t  mix  items  that  assess  behaviors  with  items  that  assess 
affective  (i.e.,  emotional)  responses. 

4.  Avoid  double-barreled  items;  each  item  should  address  a  single  issue. 

5.  Avoid  leading  questions  that  may  bias  responses. 

6.  Avoid  items  expected  to  have  little  variance  (i.e.,  show  little  difference  between  respondents). 

7.  Carefully  consider  the  use  of  negatively  worded  items. 

8.  Use  scales  appropriate  to  the  construct  you  are  measuring. 

9.  Consider  the  use  of  the  data  generated  from  the  item.  Only  use  items  that  will  produce  useful 
results. 

10.  Consider  the  key  research  questions  that  need  to  be  answered. 
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To  ensure  the  items  developed  for  this  survey  followed  these  guidelines,  all  item  writers  were  trained  on 
how  to  write  effective  items  before  item  development  began.  The  training  covered  the  following  topics: 
the  project  goals,  the  target  audience,  and  the  research  questions  to  be  addressed  by  the  project.  Item 
writers  were  trained  on  the  item  development  process  as  well  as  the  guiding  principles  for  developing 
survey  items. 

Response  Format 

During  item  development,  careful  consideration  must  be  given  to  the  response  scales  used  for  closed- 
ended  items.  Most  often  response  scales  have  five  options.  The  use  of  fewer  response  options  limits  the 
ability  to  assess  differences  between  the  respondents  (Hinkin,  1998).  The  use  of  additional  response 
options  may  be  warranted  for  some  items,  such  as  those  used  to  reflect  the  respondent’s  background. 

Scale  response  options  should  be  balanced  and  appropriate  for  the  content  of  the  question  to  aid  in 
interpretation  of  the  results.  Additionally,  we  did  not  want  to  use  the  standard  Likert-type  agreement 
scales  ( Strongly  Agree  to  Strongly  Disagree)  for  the  survey  items,  unless  there  were  no  other  acceptable 
options.  Items  with  scales  targeted  to  the  underlying  construct  often  provide  a  better  measure  of  the 
construct  than  items  using  an  agreement  scale  (Fowler,  2002). 

For  the  SOF  LCNA  survey,  item  scaling  occurred  in  conjunction  with  item  development,  and  the  scales 
were  developed  to  reflect  the  content  of  the  item.  For  example,  one  item  asked  “how  effective  was  the  pre¬ 
deployment  training  you  received  on  using  interpreters”;  thus,  the  response  options  were:  not  effective, 
slightly  effective,  moderately  effective,  effective,  very  effective.  This  is  in  contrast  to  using  an  agreement 
scale  for  an  item  such  as  “pre-deployment  training  received  on  using  interpreters  was  effective.”  Scales 
targeted  to  the  construct  allow  for  a  more  consistent  and  meaningful  interpretation  of  the  results.  Targeted 
scales  were  used  whenever  appropriate  and  feasible. 

Various  types  of  scales  were  used  in  the  present  project,  including:  categorical  scales,  Likert-type  scales, 
and  client-specific  scales.  In  addition,  open-ended  items  were  included,  allowing  respondents  to  provide 
responses  in  their  own  words.  Different  scales  were  used  in  order  to  obtain  the  most  relevant  information 
from  each  item  asked. 

Item  and  Scale  Review 

Individuals  who  are  knowledgeable  of  the  project  purpose  and  the  target  audience  should  review  all  of  the 
items  and  their  corresponding  scales.  A  review  of  the  items  should  focus  on  evaluating  the  content  of  the 
items  to  determine  if  they  are  related  to  the  project.  Additionally,  the  items  should  be  reviewed  to  ensure 
the  wording  is  suitable  for  the  target  audience  (Fowler,  2002). 

Therefore,  the  SOF  LCNA  survey  items  and  scales  were  subjected  to  a  series  of  reviews  by  internal  and 
external  expert  reviewers.  The  items  were  reviewed  for  appropriateness  of  item  content  for  the  audience, 
relevance  of  the  response  scale,  and  general  readability.  Revisions  were  then  made  to  the  items  based  on 
these  reviews.  A  complete  description  of  survey  topics  can  be  found  in  Appendix  F. 

Survey  Development  Branching 

As  items  were  generated  they  were  entered  in  Qualtrics,  a  web-survey  design  program.  Qualtrics  allowed 
creation  of  the  complex  topic  and  item  branching  logic  required  for  this  survey.  The  branching  logic 
reduced  the  item  burden  for  respondents  by  sending  them  to  topic  areas  based  on  their  reported 
experience.  For  example,  only  respondents  who  indicated  that  they  had  deployed  within  their  AOR 
received  questions  regarding  the  use  of  language  and  culture  on  deployments  within  their  AOR. 

With  the  interests  of  the  client  and  the  perspectives  of  the  different  participants  groups  in  mind,  a  list  of 
topic  areas  was  created  for  each  possible  participant  group  in  the  survey.  For  the  complete  list  of  possible 
survey  participants  and  their  respective  survey  topic  areas,  see  Appendix  G.  Therefore,  participants  for 
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certain  groups  only  received  items  in  areas  deemed  to  be  appropriate  for  their  group  and  the  order  was 
standardized.  Additionally,  participants  were  branched  past  sections  if  they  did  not  have  relevant 
experience.  After  the  branching  planning  and  maps  were  completed,  branching  logic  was  applied  both  to 
the  background  structure  (i.e.,  by  topic  area)  of  the  survey  and  within  each  topic  area  (i.e.,  at  the  item 
level)  in  Qualtrics. 

Sample  Selection 

Individuals  should  be  invited  to  participate  in  a  survey  based  on  the  alignment  of  their  background  and 
experience  with  the  project  purpose.  Since,  the  purpose  of  this  project  is  to  identify  the  language  and 
culture  needs  for  SOF  personnel,  members  of  the  SOF  community  were  targeted  for  participation.  It  was 
expected  that  some  survey  distribution  channels  could  potentially  lead  to  participation  from  non-SOF 
respondents;  therefore,  the  survey  was  designed  to  screen  out  participants  whose  perspectives  were  not 
being  targeted  as  part  of  the  data  collection  effort.  To  prevent  non-SOF  participants  from  taking  the 
survey  as  SOF-participants,  the  design  allowed  them  to  complete  a  version  of  the  survey,  instead  of  other 
options.  The  data  collected  from  non-SOF  participants  is  beyond  the  scope  of  this  project. 

Increasing  Response  Rate 

In  order  to  increase  the  likelihood  that  survey  respondents  are  representative  of  the  target  audience,  it  is 
important  to  encourage  all  eligible  participants  to  respond.  Identification  of  a  sponsor  for  the  survey  is 
one  method  for  increasing  the  number  of  respondents  (Simsek  &  Veiga,  2001).  This  sponsorship  can  be 
provided  from  high  ranking  personnel  in  the  organization.  For  this  reason,  support  for  the  survey  was 
sought  from  Admiral  Eric  T.  Olson,  Commander,  U.S.  Special  Operations  Command.  On  30  October 
2009,  Admiral  Olson  indicated  his  support  for  the  survey,  and  encouraged  participation  in  the  survey 
through  a  memo  issued  to  commanders  throughout  USSOCOM  (see  Appendix  H). 

Responses  to  surveys  are  also  increased  by  publicizing  the  survey  and  sending  reminders  to  the  target 
audience.  By  continuously  monitoring  responses  to  the  survey,  response  rates  from  specific  components 
of  the  target  audience  can  be  improved  by  sending  reminders  tailored  to  them  (Rogelberg,  2006).  The 
SOF  LCNA  survey  was  publicized  through  several  means.  Emails  were  sent  throughout  the  survey 
administration  and  survey  links  were  posted  on  AKO,  DKO,  and  SOFTS  web-pages.  Additionally,  an  SF 
command  tasker  was  sent  out  to  all  USASFC  personnel,  extending  the  survey  end  date  past  the  original 
13  November  date,  to  ensure  all  personnel  had  the  opportunity  to  participate.  A  timeline  of  the  survey 
recruitment  process  is  provided  in  Figure  5. 

It  is  also  important  to  encourage  respondents  to  complete  the  entire  survey.  This  can  be  accomplished  by 
limiting  the  number  of  items  in  the  survey  and  ensuring  that  participants  only  receive  items  relevant  to 
their  background  (Rogelberg,  2006).  Throughout  the  SOF  LCNA  survey,  branching  logic  was  used  to 
direct  respondents  to  items  relevant  to  their  background  and  experiences. 

Surx’ey  Review 

Once  the  survey  was  programmed  in  Qualtrics,  the  branching  and  display  logic  were  tested.  Reviewers 
completed  sample  responses  as  if  they  were  a  member  of  the  target  audience  to  ensure  they  were  directed 
to  the  correct  pages,  received  the  items  relevant  to  the  respondent,  and  the  correct  response  options  were 
displayed.  This  review  process  was  completed  for  each  possible  survey  participant  perspective  and  each 
possible  branching  response  in  order  to  ensure  full  functionality.  Additionally,  the  client  and  other  subject 
matter  experts  reviewed  the  functionality  of  the  survey  to  ensure  the  branching  for  each  item  was  relevant 
and  accurately  represented  the  vocabulary  of  the  target  audience. 

Features  in  Qualtrics  also  allowed  for  the  generation  of  random  test  responses  to  the  survey.  A  test  sample 
of  over  500  responses  was  created  to  further  ensure  functionality  and  allowed  researchers  to  preview  the 
data  output  and  reporting  functions  available  in  Qualtrics. 
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Respondent  Tracking 

Qualtrics  was  used  to  administer  the  survey  and  manage  the  data.  Through  Qualtrics,  reports  were 
generated  each  day  that  the  survey  was  available  for  respondents  to  complete.  These  reports  indicated  the 
total  number  of  respondents  to  the  survey.  Features  within  Qualtrics  were  also  used  to  assess  the  number 
of  responses  from  each  component  of  the  target  audience.  Creating  item  summary  reports  for  the  client 
provided  the  information  needed  to  modify  the  survey  recruitment  process  in  an  effort  to  increase 
response  rates  across  the  SOF  community. 
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Figure  5.  Survey  Recruitment  Timeline 
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SECTION  V:  SURVEY  ANALYSIS 


Analysis  Preparations 

At  the  completion  of  data  collection,  the  survey  data  were  downloaded  from  Qualtrics  into  a  comma 
separated  values  (CSV)  spreadsheet.  Due  to  the  large  number  of  responses  and  items  on  the  survey, 
several  different  programs  were  used  for  analysis  including:  Qualtrics,  Microsoft  Excel  and  CSV 
spreadsheets,  and  SYSTAT.  Qualitative  data  from  the  survey  were  analyzed  in  Microsoft  Excel.  The  use 
of  different  programs  was  also  necessary  because  each  had  its  own  limitations  that  were  often  resolved  by 
the  use  of  another.  The  drawback  to  using  multiple  programs  was  the  amount  of  data  preparation 
necessary  to  ensure  each  file  contained  the  appropriate  amount  of  information.  This  was  especially 
important  with  the  use  of  Excel  and  SYSTAT  because  of  the  data  size  restrictions  in  these  programs. 

Data  Cleaning 

In  order  to  ensure  that  the  survey  data  were  an  accurate  reflection  of  the  participants’  responses,  it  was 
necessary  to  recode  (i.e.,  assign  new  quantitative  values)  some  of  the  questions.  For  background  questions 
where  an  exhaustive  list  of  options  was  prohibitive,  respondents  were  given  the  option  to  select  “Other” 
and  type  in  a  response  that  best  reflected  their  background.  Therefore,  it  was  necessary  to  assign  new 
values  to  these  text  responses  in  order  to  appropriately  classify  the  respondents  and  include  them  in 
subsequent  analyses. 

One  area  where  this  approach  was  necessary  was  for  participants  who  indicated  they  were  part  of  the  SOF 
community  but  selected  “Other”  because  they  did  not  see  an  option  listed  that  described  their  current  role 
within  the  SOF  community.  It  is  imperative  to  be  able  to  accurately  describe  the  respondent’s  role  in  the 
SOF  community  to  evaluate  any  differences  in  responses  that  may  occur  across  the  SOF  community.  For 
this  item,  the  client  was  consulted  and  asked  to  evaluate  the  responses  to  clearly  establish  their  role  within 
SOF.  The  client’s  feedback  was  then  used  to  assign  new  values  to  these  responses,  thereby  placing  them 
with  their  correct  role.  For  example,  a  number  of  SOF  participants  who  indicated  “other”  as  their  current 
role  in  the  SOF  community  identified  themselves  as  Civil  Affairs  or  Psychological  Operations  personnel. 
These  are  both  valid  roles  in  the  SOF  community;  based  on  other  background  information  such  as  MOS 
and  deployment  experience,  these  individuals  were  recoded  as  SOF  operators  or  SOF  unit  leaders. 

For  the  additional  areas  where  this  approach  was  taken,  the  client’s  input  was  not  needed.  There  were  21 
questions  providing  enough  unique  responses  to  support  the  need  to  create  values  to  represent  those 
responses  and  update  the  data  to  reflect  these  additions.  For  each  question,  the  text  responses  were 
reviewed  to  determine  how  many  new  values  were  needed  to  accurately  portray  the  respondents’ 
backgrounds.  If  a  response  was  submitted  by  more  than  one  respondent,  a  value  was  assigned  to  the 
response  and  the  data  was  updated.  For  example,  USASOC  participants  were  asked  to  indicate  their 
MOS,  and  20  respondents  who  indicated  “other”  wrote  in  68 W  as  their  MOS.  Due  to  the  large  response 
for  this  MOS,  a  68 W  option  was  created  and  the  appropriate  respondents  were  recoded.  Additional  items 
where  responses  were  recoded  included  language  background  and  SOF  mission  type. 

Coding  Comments 

The  survey  offered  many  opportunities  for  respondents  to  answer  open-ended  questions,  providing 
comments  from  their  perspectives  on  various  issues.  For  these  questions,  the  respondent  was  simply 
asked  to  type  in  any  comments  they  had  relating  to  the  question.  Since  these  items  resulted  in  data  that 
were  in  text  format  only,  the  data  needed  to  be  coded  before  it  could  be  analyzed. 

Coding  Process 

Two  coders  collaborated  on  each  survey  content  area  containing  open-ended  comments.  Each  coder 
independently  read  the  respondents’  comments  related  to  their  topic  area  and  noted  the  content  of  those 
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comments.  The  coders  then  compared  the  combined  comment  lists  with  the  existing  content  codes 
developed  for  the  focus  group  data  (see  Focus  Group  Analysis  Section).  If  the  content  area  was  captured 
by  an  existing  code,  it  was  used  as  a  comment  code.  The  remaining  content  areas  were  reviewed  for  major 
themes.  The  coders  compared  their  list  of  themes  and  came  to  a  consensus  on  the  themes  represented  by 
the  data  and  the  appropriate  names  for  the  content  codes  that  would  represent  the  themes.  Once  the  code 
list  was  finalized,  the  coders  used  both  the  focus  group  codes  and  the  newly  developed  codes  for  content 
coding.  In  a  Microsoft  Excel  spreadsheet,  coders  independently  assigned  the  number  associated  with  the 
relevant  code(s)  in  a  column  next  to  each  survey  comment. 

Quality  Assurance 

Quality  assurance  was  ensured  using  the  same  methods  described  in  Section  III:  Focus  Group  Analysis. 
All  coders  were  trained  on  the  coding  process.  Multiple  coders  were  used,  and  rater  agreement  was 
calculated  after  all  comments  were  coded.  Absolute  agreement  was  calculated  at  the  completion  of  coding 
for  each  qualitative  item.  The  coders  then  met  to  come  to  a  consensus  on  any  areas  of  disagreement. 

Quantitative  Items 

Items  that  presented  the  respondent  with  a  set  of  response  options  were  analyzed  using  the  appropriate 
descriptive  or  inferential  statistics.  Descriptive  statistics  (e.g.,  frequencies  and  averages)  were  calculated 
to  provide  further  information  about  participants  and  the  distribution  of  their  responses.  Additionally, 
inferential  statistics  (e.g.,  analysis  of  variance,  /-tests)  were  used  to  determine  if  any  observed  differences 
were  likely  to  exist  in  the  broader  population  of  interest.  The  specific  analyses  used  for  quantitative  items 
differ  from  topic  to  topic,  depending  on  the  item  types  involved  and  information  requirements.  For 
example,  in  the  LCNA  Training  Emphasis  Report  ( Technical  Report  #201001 1005),  individual 
quantitative  items  were  analyzed  using  frequencies  and  averages.  In  order  to  compare  group  differences 
(e.g.,  civil  affairs  v.  psychological  operations)  on  training  emphasis,  items  were  analyzed  using  analysis 
of  variance  and  /-tests.  Each  issue  report  will  provide  a  brief  description  of  the  analysis  protocol. 
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ABOUT  SWA  CONSULTING  INC. 

SWA  Consulting  Inc.  (formerly  Surface,  Ward,  and  Associates)  provides  evidence-based  solutions  for 
clients  using  the  principles  and  methods  of  industrial/organizational  (I/O)  psychology.  Since  1997,  SWA 
has  advised  and  assisted  corporate,  non-profit  and  governmental  clients  on: 

•  Training  and  development 

•  Performance  measurement  and  management 

•  Organizational  effectiveness 

•  Test  development  and  validation 

•  Program/training  evaluation 

•  Work/job  analysis 

•  Needs  assessment 

•  Selection  system  design 

•  Study  and  analysis  related  to  human  capital  issues 

•  Metric  development  and  data  collection 

•  Advanced  data  analysis 

One  specific  practice  area  is  research  and  consulting  on  foreign  language  and  culture  in  work  contexts.  In 
this  area,  SWA  has  conducted  numerous  projects,  including  language  assessment  validation  and 
psychometric  research;  evaluations  of  language  training,  training  tools,  and  job  aids;  language  and  culture 
focused  needs  assessments  and  job  analysis;  and  advanced  analysis  of  language  research  data. 

Based  in  Raleigh,  NC,  and  led  by  Drs.  Eric  A.  Surface  and  Stephen  J.  Ward,  SWA  now  employs  close  to 
twenty  I/O  professionals  at  the  masters  and  PhD  levels.  SWA  professionals  are  committed  to  providing 
clients  the  best  data  and  analysis  with  which  to  make  solid  data-driven  decisions.  Taking  a  scientist- 
practitioner  perspective,  SWA  professionals  conduct  model-based,  evidence-driven  research  and 
consulting  to  provide  the  best  answers  and  solutions  to  enhance  our  clients’  mission  and  business 
objectives. 

For  more  information  about  SWA,  our  projects,  and  our  capabilities,  please  visit  our  website  (www.swa- 
consulting.com)  or  contact  Dr.  Eric  A.  Surface  (esurface @ swa-consulting.com)  or  Dr.  Stephen  J.  Ward 
(sward@swa-consulting.com). 
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APPENDIX  A:  ABOUT  THE  LCNA  PROJECT 

In  2003-2004,  the  Special  Operations  Forces  Culture  and  Language  Office  (SOFCLO;  formerly,  SOFLO) 
sponsored  the  SOF  Language  Transformation  Strategy  Needs  Assessment  Project  to  inform  the 
development  of  a  language  transformation  strategy  in  response  to  a  GAO  report  (2003).  This  SOF 
Language  Transformation  Strategy  Needs  Assessment  Project  collected  current-state  information  about 
language  usage,  proficiency,  training,  and  policy  issues  (e.g.,  Foreign  Language  Proficiency  Pay,  FLPP) 
from  SOF  personnel,  SOF  unit  leaders,  and  other  personnel  involved  in  SOF  language.  The  project  used 
multiple  data  collection  methods  and  provided  the  SOFCLO  with  valid  data  to  develop  a  comprehensive 
language  transformation  strategy  and  advocate  for  the  SOF  perspective  on  language  issues  within  the 
DoD  community. 

In  a  continuing  effort  to  update  knowledge  of  language  and  culture  needs  while  informing  strategic  plan 
development,  the  SOFCLO  commissioned  the  2009  SOF  Language  and  Culture  Needs  Assessment 
Project  (LCNA)  to  reassess  the  language  and  culture  landscape  across  the  United  States  Special 
Operations  Command  (USSOCOM)  and  develop  a  strategy  for  the  next  five  years.  Data  were  collected 
between  March  and  November,  2009  from  personnel  in  the  SOF  community,  including  operators  and 
leaders.  Twenty-three  focus  groups  were  conducted  between  March  and  June,  2009.  A  comprehensive, 
web-based  survey  designed  to  gather  information  from  both  operators  and  leaders  in  the  SOF  community 
was  launched  on  26  October  and  closed  on  24  November,  2009. 

This  project’s  findings  will  be  disseminated  through  reports  and  briefings  (see  Appendix  A,  Figure  1  for 
an  overview).  Two  foundational  reports  will  document  the  methodology  and  participants  associated  with 
this  project.  The  remaining  reports  will  be  organized  in  three  tiers.  The  specific  reports  in  each  of  these 
tiers  will  be  determined  and  contracted  by  the  SOFCLO.  As  originally  planned,  twenty-five  Tier  I  Reports 
will  focus  on  specific,  limited  issues  [e.g.,  Inside/Outside  Area  of  Operations  (AOR)  Use  of  Cultural 
Knowledge,  Inside  AOR  Use  of  Language].  Tier  II  reports  will  integrate  and  present  the  most  important 
findings  across  related  Tier  I  reports  (e.g.,  Use  of  Language  and  Culture  on  Deployment).  Most,  but  not 
all,  Tier  I  reports  will  roll  into  Tier  II  reports.  One  Tier  III  Report  will  present  the  most  important 
findings,  implications,  and  recommendations  across  all  topics  explored  in  this  project.  The  remaining  Tier 
III  reports  present  findings  for  specific  SOF  organizations  [e.g.,  Air  Force  Special  Operations  Command 
(AFSOC),  Special  Forces  (SF)  Command].  All  Tier  III  reports  will  be  associated  with  a  briefing.  As 
mentioned,  the  additional  reports  will  be  determined  by  the  SOFCLO  and  may  differ  from  what  was 
originally  planned. 

In  June,  2009,  the  GAO  reported  that  the  Department  of  Defense  is  making  progress  toward  transforming 
language  and  regional  proficiency  capabilities  but  still  does  not  have  a  strategic  plan  in  place  to  continue 
development  that  includes  actionable  goals  and  objectives.  The  findings  from  this  study  can  be  used  by 
the  SOFCLO  and  leaders  at  USSOCOM  to  continue  strategic  planning  and  development  in  this  area. 

This  project  design,  logistics,  data  collection,  initial  analysis  and  first  eight  reports  of  this  project  were 
conducted  by  SWA  Consulting  Inc.  (SWA)  under  a  subcontract  with  SRC  (SR20080668  (K142);  Prime  # 
N65236-08-D-6805).  The  additional  reports  mentioned  above  are  proposed  for  the  future  (TBD  by  the 
SOFCLO).  For  questions  or  more  information  about  the  SOFCLO  and  this  project,  please  contact  Mr. 
Jack  Donnelly  (john.donnelly@socom.mil).  For  specific  questions  related  to  data  collection  or  reports 
associated  with  this  project,  please  contact  Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr. 
Reanna  Poncheri  Harman  (rpharman@swa-consulting.com)  with  SWA  Consulting  Inc. 
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Appendix  A,  Figure  1 .  Report  Overview 


Foundation  Reports  LkJ  Tier  I  Reports  Current  Contract 


1.  Methodology  Report 

2.  Participation  Report 


3.  Admiral  Olson's  Memo 

4.  Training  Emphasis:  Language  and  Culture 

5.  Command  Support  of  Language:  Grading  the 
Chain  of  Command 

6.  SOFCLO  Support 

7.  Inside/Outside  AOR  Use  of  Cultural  Knowledge 

8.  Team  Composition 


Tier  I  Reports  Proposed  for  Future 
(TBD  by  SOFCLO) 


9.  Inside  AOR  Use  of  Language 

10.  Outside  AOR  Use  of  Language 

11.  Mission-Specific  Use  of  Interpreters 

12.  General  Use  of  Interpreters 
13.09L 

14.  DLPT 

15.  OPI 

16.  Selection  Tests:  DLAB 

17.  Initial  Acquisition  Training 

18.  Sustainment/Enhancement  Training 

19.  Culture  Training 

20.  Immersion 

21.  Language  Resources,  Technology  &  Self-Study 

22.  Foreign  Language  Proficiency  Bonus 

23.  Non-monetary  Incentives 

24.  Command  Support  of  Language:  Other 
Barriers/Organizational  Support 

25.  Force  Motivation  for  Language 

26.  Leader-Specific  Issues  Report 

27.  CLPM-Specific  Issues  Report 


Tier  II  Reports  Proposed  for  Future 
(TBD  by  SOFCLO) 


28.  Use  of  Language  and  Culture  on  Deployment 

29.  Use  of  Interpreters 

30.  Team  Composition  and  Capability 

31.  Testing/Metrics 

32.  Current  State  of  Language  Training 

33.  Language  Training  Guidance 

34.  Culture  Training  Guidance 

35.  Incentives/Barriers 

I 


Tier  III  Reports  Proposed  for  Future 
(TBD  by  SOFCLO) 


36.  Overall  Picture:  Conclusions  and 
Recommendations 

37.  AFSOC 

38.  MARSOC 

39.  WARCOM 

40.  SF  Command 

41.  CA 

42.  PSYOP 

43.  Seminar  Briefing(s) 


Note :  Foundation  reports  are  referenced  by  every  other  report.  Colors  represent  Tier  I  reports  that  roll  (integrate)  into  an  associated  Tier  II  report.  Reports  in  black  are  final  reports  on  the  topic  but 
may  be  cited  by  other  reports.  Tier  II  reports  roll  into  the  Tier  III  reports.  All  Tier  III  reports  include  an  associated  briefing. 
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APPENDIX  B:  GROUND  RULES  FOR  SOF  FOCUS  GROUP  SESSIONS 

Ground  rules  are  provided  to  help  the  focus  group  flow  smoothly.  These  ground  rules  must  be  posted  so 
that  all  focus  group  participants  can  see  them  or  distributed  to  all  participants. 

1.  Do  not  include  any  classified  information/details  when  discussing  critical  incidents  of  language 
usage  from  the  field. 

2.  Every  participant’s  view  is  important,  and  everyone  has  equal  say  in  the  discussions.  Everyone’s 
view  is  of  value.  Listening  is  as  important  as  speaking. 

3.  Focus  group  participants  should  treat  all  opinions  and  views  expressed  by  other  participants 
inside  the  focus  group  as  privileged.  Confidentiality  is  very  important. 

4.  Differences  of  opinion  are  good. 

5.  There  will  be  no  personal  attacks  on  anyone.  If  you  disagree  with  a  comment,  state  your  point 
without  making  it  personal. 

6.  We  must  only  speak  from  our  personal  experiences  and  observations  and  not  from  hearsay. 

7.  We  must  all  act  with  respect  and  professionalism  at  all  times  during  the  focus  group. 

8.  Everyone  must  actively  participate  in  the  discussions,  and  no  one  person  should  dominate. 

9.  In  order  to  respect  others,  there  should  be  only  one  person  speaking  at  a  time,  and  there  cannot  be 
any  side  conversations  during  the  discussions.  If  you  wish  to  speak,  get  the  moderator’s  attention 
(for  example,  raise  your  hand). 

10.  To  protect  your  confidentiality,  the  names  of  participants  will  not  be  used  or  reported  in  the 
official  records.  After  transcription,  the  audio  files  will  be  erased  to  further  protect  participants. 
The  recording  devices  are  not  activated  until  after  the  introductions.  Refer  to  other  group 
members  by  their  participant  number  only. 
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APPENDIX  C:  FOCUS  GROUP  PARTICIPANT  SURVEY 


SOF  Language  and  Culture  Needs  Assessment 

Focus  Group  Participant  Survey 


SWA 

Consulting 


Please  Indicate  Your  SOF  Component:  _ 

Please  Indicate  Your  MOS/NEC/AFSC:  _ 

What  is  your  primary  foreign  language?  _ 

Please  indicate  your  most  recent  proficiency  scores  in  this  language: 


0 

0+ 

1 

1+ 

2 

2+ 

3  (or  above) 

Listening 

o 

o 

o 

o 

o 

o 

o 

Reading 

o 

o 

o 

o 

o 

o 

o 

Speaking 

o 

o 

o 

o 

o 

o 

o 

What  is  your  secondary  foreign  language  (if  applicable)? 

Please  indicate  your  most  recent  proficiency 

scores 

in  this  language: 

0 

0+ 

1 

1+ 

2 

2+ 

3  (or  above) 

Listening 

o 

o 

o 

o 

O 

o 

o 

Reading 

o 

o 

o 

o 

o 

o 

o 

Speaking 

o 

o 

o 

o 

o 

o 

o 

Please  provide  any  additional  thoughts  or  comments  about  the  content  we  discussed  in  Block  1  (Language  and  Culture): 


Copyright  ©  2004-2009  SWA  Consulting  Inc.  All  rights  reserved. 
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SOF  Language  and  Culture  Needs  Assessment 

Focus  Group  Participant  Survey 


SWA 

Consulting 


Please  provide  any  additional  thoughts  or  comments  about  the  content  we  discussed  in  Block  2  (Training): 


Please  provide  any  additional  thoughts  or  comments  about  the  content  we  discussed  in  Block  3  (Activity): 


For  your  primary  mission,  which  is 


(e.g.,  UW),  rate  the  importance  of  the  following  to  mission  success: 


Not  Important 

(I  can  do  mv 
mission  without  it) 

Imp  ortant 

Critically  Important 

(1  can’t  do  mv 
mission  without  it) 

Language 

(without  Interpreters ) 

o 

o 

O 

o 

o 

Culture 

o 

o 

o 

o 

o 

Interpreters 

o 

o 

o 

o 

o 

Copyright  ©  2004-2009  SWA  Consulting  Inc.  All  rights  reserved.  Booklet:  «Booklets»  Page:  002 
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APPENDIX  D:  FULL  PROTOCOL  FOCUS  GROUP  INTERVIEW  GUIDE 


This  script  will  guide  you  in  moderating  the  focus  group(s)  that  you  are  assigned. 


Sample  Timeline  &  Script 

8:15/13:15 

[Participants  arrive,  sign-in,  receive  focus  group  guidelines,  and  name  plates  (participant 
numbers  only);  name  plates  will  have  a  number  1  to  10  for  the  note  takers  and  other 
participants.  Give  them  a  copy  of  the  first  question  block  and  the  participant  survey.] 
8:30/13:30 

SOFLO  Liaison:  Please  take  a  seat  so  we  can  begin.  Good  morning/afternoon!  My  name  is 
[SOFLO  Liaison  name],  and  I  am  here  on  behalf  of  the  SOF  Language  Office.  We  want  to  know 
your  views  about  language  proficiency,  training,  and  related  issues.  The  information  that  you 
provide  will  help  us  to  understand  the  issues  from  the  SOF  Operators’  perspective.  Your  input 
will  help  us  develop  a  comprehensive  language  and  culture  strategy  for  SOF.  The  SOF  language 
and  culture  strategy  influences  how  and  what  type  of  training  and  testing  are  done;  how  incentive 
pays  are  structured,  etc.  Thank  you  for  taking  time  to  talk  with  us  today  and  caring  about  SOF 
language.  Now,  I  want  to  introduce  the  person  who  will  be  conducting  this  focus  group 
[Moderator  Name]. 

8:35/13:35 

Moderator :  Good  morning/afternoon!  I  want  to  echo  what  [SOFLO  Liaison  Name]  said.  Your 
input  is  important  to  our  project,  and  we  appreciate  your  time.  We  will  be  using  the  information 
you  provide  to  determine  current  language  and  culture  requirements  and  other  issues  to  inform 
future  policy.  Your  opinions  and  experience  play  an  important  role  in  developing  the  SOF 
language  and  culture  strategy. 

When  you  arrived,  you  received  a  list  of  ground  rules  designed  to  ensure  the  session  runs 
smoothly.  [Go  over  a  few  of  the  rules.]  If  you  have  a  comment,  please  get  my  attention  by 
raising  your  hand  or  by  some  other  means  that  doesn’t  disrupt  the  conversation. 


Unclassified  version 
No  names  or  ranks 

Speak  one  at  a  time;  Get  my  attention 

Speak  clearly,  loudly  and  toward  the  microphone 

Military  Jargon  and  Acronyms 


There  are  no  “right  or  wrong’’  answers  to  our  questions.  We  are  interested  in  your  honest  thoughts 
and  opinions.  You'll  notice  that  we  are  taking  notes  of  our  discussion,  and  we  will  be  recording 
this  focus  group  so  that  the  transcripts  can  be  analyzed.  To  protect  your  confidentiality,  the 
introductions  are  not  recorded  and  no  names  or  group  ids  are  assigned  to  the  transcripts.  Once  the 
“media  files”  are  transcribed,  it  will  be  erased  (to  prevent  voice  recognition). 


As  a  psychologist,  it’s  important  to  me  to  protect  your  confidentiality.  I  want  to  assure  you  that 
the  transcript  we  make  of  our  discussion  will  not  be  used  for  any  purpose  other  than  research  and 
analysis.  No  one  involved  in  the  focus  group  will  be  identified  by  name.  The  recorders  will 
not  be  activated  until  after  the  introductions.  When  referring  to  yourself  or  others,  please 
use  only  their  participant  numbers.  Please  speak  one  at  a  time  and  regard  the  note  taking  as 
simply  an  extension  of  my  (the  moderator’s)  memory. 

The  format  of  today’s  session  is  on  your  handout.  [Review  agenda].  The  times  are  a  rough 
estimation.  The  segments  will  vary  in  time. 
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[The  moderator  starts  the  introduction  by  talking  about  himself/herself.] 

I'd  like  to  begin  by  going  around  the  circle,  starting  at  my  right,  and  letting  each  of  you  introduce 
yourself  briefly  by  telling  us  the  following  things  [write  on  first  page  of  the  board]: 

•  MOS/NEC  (naval  enlisted  specialty  code)/AFSC  (Air  Force  Specialty  Code) 

•  SOF  Component 

•  Years  in  Service/SOF 

•  Language(s)  and  where  trained  in  language 

[Each  participant  speaks] 

Thank  you. 

Now,  I  think  we  all  know  a  little  bit  about  each  other,  I'd  like  to  get  started  with  the  questions. 

[Moderator  will  ask  questions  and  summarize  after  each  question  as  discussed.] 

8:50/13:50 

[Moderator  should  remind  the  participants  to  communicate  critical  incidents  in  such  a  way 
that  they  are  unclassified.] 

First  Question  Block 

Part  1:  Language  Usage  &  Capability  Requirements: 

Ql:  “Tell  us  about  a  situation  where  having  language  skills  and  capabilities  contributed 
to  mission  success?” 

•  What  was  the  proficiency  level?  How  many  on  the  team  had  language 
capability? 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  Was  the  mission  inside  or  outside  your  AOR? 

•  Did  you  use  an  interpreter?  (Was  the  interpreter  a  contractor  or  military 
linguist?) 

•  Did  you  do  pre-mission  language  prep  ? 

•  Did  you  use  job  aids  (e.g.,  Kwikpoint)  to  help  prepare?  If  yes,  what’s  your 
evaluation  of  the  job  aid? 

Q2:  “Tell  us  about  a  situation  where  not  having  sufficient  language  skill  or  capabilities 
was  a  challenge  for  the  mission  or  degraded  the  mission  outcome?” 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  How  did  it  take  away  from  mission  accomplishment? 

•  How  did  you  overcome  not  having  sufficient  language  capabilities? 

•  How  would  having  better  language  skills  enhance  the  mission  ? 

Q3:  “Which  SOF  mission  activities  require  language  capabilities?  Please  be  specific  in 
terms  of  language  skill  type  (reading,  listening,  speaking,  writing),  activity  (e.g.,  training 
troops  on  weapon  maintenance,  negotiation,  etc.),  and  criticality  of  the  activity.  Be  as 
descriptive  as  possible  when  talking  about  the  specific  activities.  ” 
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Q4:  “What  types  of  language  capabilities  should  the  team  have?  Should  everyone  on  the 
team  (or  only  a  few)  have  some  capability?  What  levels  of  capability  should  the  team 
have?  Should  there  be  multiple  languages  on  a  team?” 

Part  2:  Cultural  Knowledge  and  Awareness  Requirements: 

Q5:  “ Tell  us  about  a  situation  where  having  cultural  knowledge  and  awareness 
contributed  to  mission  success?”  (Provide  as  much  detail  as  possible ) 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  Did  you  have  any  pre-mission  culture  training? 

Q6:  “Tell  us  about  a  situation  where  not  having  sufficient  cultural  knowledge  and 
awareness  was  a  challenge  for  the  mission  or  degraded  the  mission  outcome?” 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved  ? 

•  How  did  it  take  away  from  mission  accomplishment? 

•  How  did  you  overcome  not  having  sufficient  cultural  knowledge  and 
awareness? 

•  How  would  having  better  cultural  knowledge  and  awareness  enhance  the 
mission  ? 

Q7:  “Which  SOF  mission  activities  require  cultural  knowledge  and  awareness?  Please 
be  specific  in  terms  of  activity  (e.g.,  training  troops  on  weapon  maintenance,  negotiation, 
etc.)  and  criticality  of  the  activity.  Be  as  descriptive  as  possible  when  talking  about  the 
specific  activities.  ” 

[Moderator  summarizes  first  block,  reminds  participants  to  add  any  additional  comments 
from  the  first  block  on  the  survey,  and  calls  a  short  break.] 

9:50/14:50 

Break  1  (10  minutes) 

10:00/15:00 

Second  Question  Block  (Training): 

Q8:  “Where  do  you  think  language  and  culture  training  should  be  provided/offered  in  the 
development  of  a  SOF  operator?  Please  think  about  this  question  across  the  entire  career 
lifecycle  and  in  terms  of  all  forms  of  training  (e.g.,  pre-deployment).  ” 

•  Should  institutional,  involuntary  training  be  provided  for  [their  SOF 
component]  ? 

Q9:  “We  are  interested  in  your  experiences  with  language/culture  training.  Tell  us  about 
the  most  effective  language/culture  training  in  which  you  have  participated.  Be  sure  to 
consider  all  forms  of  training  (e.g.,  pre-deployment).  Be  descriptive  in  terms  of  why  it 
was  effective.  ” 

Q10:  “Tell  us  about  the  least  effective  language/culture  training  in  which  you  have 
participated.  Be  sure  to  consider  all  forms  of  training  from  initial  acquisition  to 
sustainment  and  enhancement.  Be  descriptive  in  terms  of  why  it  was  not  effective.  ” 

[Moderator  may  only  get  to  one  or  two  of  the  following  so  they  should  be  mixed  up 
across  groups.  But,  get  to  as  many  as  possible.] 
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QH:  “What  are  some  of  the  barriers  or  things  that  keep  you  from  maintaining  or 
enhancing  your  language/culture  skills?” 

Q12:  “What  things  would  motivate  you  or  make  it  easier  for  you  to  take  advantage  of 
language/culture  training  ?  ” 

Q13:  “Are  you  motivated  to  increase  your  language  proficiency  level  to  qualify  for 
FLPB  (Foreign  Language  Proficiency  Bonus)?” 

Q14:  “What  are  your  thoughts  on  language  testing?  Are  the  current  tests  an  adequate 
indicator  of  your  language  proficiency?” 

Q15:  “Can  language/culture  training  be  integrated  into  other  SOF  training?  Have  you 
participated  in  any  of  these  programs?  What  did  you  think?” 

Q16:  “ Classroom  training  and  live  environment  training  (LET)  programs  are  not  always 
available.  Is  distance  learning  (DL)  or  technology-delivered  training  (TDT)  a  viable 
option  for  training  language  and  culture?”  Give  example  if  necessary. 

Q17:  “Have  you  ever  completed  language  training  via  SOFTS?  What  are  your  thoughts 
about  SOFTS?” 

[Moderator  summarizes  second  block,  reminds  participants  to  add  any  additional 
comments  from  the  second  block  on  the  survey,  and  calls  a  short  break.] 

11:05/16:05 

Break  2  (5  minutes) 

11:10/16:10 

[This  block  has  the  potential  to  go  over  in  terms  of  time.] 

Third  Question  Block  (Activity:  Developing  a  language/culture  training  strategy): 

You  are  to  divide  into  3  teams  of  2  to  5  people  (as  assigned  by  the  moderator).  You  will 
have  20  minutes  to  develop  a  decision  briefing  to  be  given  to  your  unit  commander.  You 
should  select  a  representative  to  brief,  and  your  group  should  write  up  the  basic  points  of 
your  briefing  on  the  paper  provided.  You  will  have  5  minutes  to  brief  it.  You  will  be 
given  a  5-minute  warning  after  15  minutes  have  passed. 

Please  select  one  of  the  following: 

1.  The  Commander  of  your  SOF  Unit  asks  you  to  develop  a  strategy  and  plan  to 
transform  current  language  capabilities  and  cultural  knowledge  and  awareness  to  ensure 
mission  success.  He  wants  this  yesterday.  He  asks  you  to  prepare  a  5-minute  decision 
brief  immediately,  and  he  indicates  your  language/culture  improvement  strategy  and  plan 
should  consider/address  the  following: 

■  Mission  language/culture  requirements 

•  which  language  skills/cultural  knowledge  and  awareness  are  relevant 

•  what  proficiency  levels  are  needed 

■  Where  training  fits  into  the  career  lifecycle 
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■  The  most  effective  training  options  for  initial  acquisition  and 
sustainment/enhancement  training 

•  whether  initial  acquisition  training  needs  to  be  institutionalized  for 
all  operators 

•  should  training  at  all  levels  be  voluntary  or  involuntary 

•  The  use  and  role  of  distance  learning  and  other  technology  solutions 

■  Barriers  that  interfere  with  or  prevent  the  development  and  maintenance  of 
language  capabilities  and  cultural  knowledge  and  awareness 

•  What  are  these  barriers? 

•  How  can  they  be  removed? 

■  What  programs  would  you  recommend  to  motivate  improved  language 
proficiency  and  cultural  knowledge  and  awareness? 

2.  The  Commander  of  your  SOF  unit  asks  you  to  identify  the  top  3  actions  that  can  be  taken  to 
improve  language  capability,  cultural  knowledge  and  awareness,  and  readiness  in  your  unit.  He 
wants  this  yesterday.  He  asks  you  to  prepare  a  5-minute  decision  brief  immediately.  For  each 
action,  you  must  provide  (1)  the  language  capability,  cultural  knowledge  and  awareness,  and 
readiness  issue  the  action  impacts;  (2)  a  rationale  for  how  it  impacts  the  language/cultural  issue; 
(3)  a  rationale  for  why  it  is  the  best  option  for  resolving  the  issue;  (4)  the  specific  details  of  the 
action’s  implementation;  (5)  an  idea  of  where  the  action  fits  into  the  current  command 
language/culture  program/strategy;  and  (6)  any  other  information  you  believe  the  Commander 
needs  to  make  a  decision. 

Divide  into  3  teams  of  2  to  5  people  [depends  on  who  shows].  You  will  have  20  minutes 
to  develop  your  strategy  and  plan.  You  should  write  up  the  basic  points  to  present  to 
the  group.  You  will  have  5  minutes. 

[The  recorders  are  turned  off,  and  the  groups  work.  They  are  given  chart  paper  and 
marker.] 

[After  15  minutes,  a  5-minute  warning  is  given.] 

[Recorders  are  turned  back  on  and  each  group  presents.  The  moderator  asks  the  groups 
questions  about  their  plans  as  appropriate.] 

12:05/17:05 

[Moderator  will  summarize  entire  focus  group  session.  Be  specific  about  any  themes  that  emerged  from 
the  discussion.] 

Before  we  conclude  does  any  one  have  any  additional  comments  or  topics  related  to  language  or  culture 
that  they  would  like  to  discuss.  [Remind  participants  to  take  a  few  minutes  to  complete  the  survey.] 

[To  SOFLO  Liaison]  Do  you  have  any  thing  you  would  like  to  add? 

Thank  you  all  for  your  generosity  in  taking  the  time  out  of  your  busy  schedules  to  talk  with  us  today. 
[Moderator  ends  interview.] 
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APPENDIX  E:  ABBREVIATED  PROTOCOL  FOCUS  GROUP  INTERVIEW  GUIDE 

This  script  will  guide  you  in  moderating  the  focus  group(s)  that  you  are  assigned. 

Sample  Timeline  &  Script 

8:15/13:15 

[Participants  arrive,  receive  focus  group  guidelines,  and  the  participant  survey.  Numbered 
cards  will  be  at  each  empty  seat  and  will  be  numbered  from  1-10.  Participants  will  be 
referred  to  by  number  during  the  focus  group.] 

8:30/13:30 

SOFLO  Liaison:  Please  take  a  seat  so  we  can  begin.  Good  morning/afternoon!  My  name  is 
[SOFLO  Liaison  name],  and  I  am  here  on  behalf  of  the  SOF  Language  Office.  We  want  to  know 
your  views  about  language  proficiency,  training,  and  related  issues.  The  information  that  you 
provide  will  help  us  to  understand  the  issues  from  the  SOF  Operators’  perspective.  Your  input 
will  help  us  develop  a  comprehensive  language  and  culture  strategy  for  SOF.  The  SOF  language 
and  culture  strategy  influences  how  and  what  type  of  training  and  testing  are  done;  how  incentive 
pays  are  structured,  etc.  Thank  you  for  taking  time  to  talk  with  us  today  and  caring  about  SOF 
language.  Now,  I  want  to  introduce  the  person  who  will  be  conducting  this  focus  group 
[Moderator  Name]. 

8:35/13:35 

Moderator.  Good  morning/afternoon!  I  want  to  echo  what  [SOFLO  Liaison  Name]  said.  Your 
input  is  important  to  our  project,  and  we  appreciate  your  time.  We  will  be  using  the  information 
you  provide  to  determine  current  language  and  culture  requirements  and  other  issues  to  inform 
future  policy.  Your  opinions  and  experience  play  an  important  role  in  developing  the  SOF 
language  and  culture  strategy. 

When  you  arrived,  you  received  a  list  of  ground  rules  designed  to  ensure  the  session  runs 
smoothly.  [Go  over  a  few  of  the  rules.]  If  you  have  a  comment,  please  get  my  attention  by 
raising  your  hand  or  by  some  other  means  that  doesn’t  disrupt  the  conversation. 

Unclassified  version 
No  names  or  ranks 

Speak  one  at  a  time;  Get  my  attention 

Speak  clearly,  loudly  and  toward  the  microphone 

Military  Jargon  and  Acronyms 

There  are  no  “right  or  wrong’’  answers  to  our  questions.  We  are  interested  in  your  honest  thoughts 
and  opinions.  You'll  notice  that  we  are  taking  notes  of  our  discussion,  and  we  will  be  recording 
this  focus  group  so  that  the  transcripts  can  be  analyzed.  To  protect  your  confidentiality,  the 
introductions  are  not  recorded  and  no  names  or  group  ids  are  assigned  to  the  transcripts.  Once  the 
“media  files’’  are  transcribed,  they  will  be  erased  (to  prevent  voice  recognition). 

As  a  psychologist,  it’s  important  to  me  to  protect  your  confidentiality.  I  want  to  assure  you  that 
the  transcript  we  make  of  our  discussion  will  not  be  used  for  any  purpose  other  than  research  and 
analysis.  No  one  involved  in  the  focus  group  will  be  identified  by  name.  The  recorders  will 
not  be  activated  until  after  the  introductions.  When  referring  to  yourself  or  others,  please 
use  only  their  participant  numbers.  Please  speak  one  at  a  time  and  regard  the  note  taking  as 
simply  an  extension  of  my  (the  moderator’s)  memory. 

[Moderator  will  ask  questions  and  summarize  after  each  question  as  discussed.] 
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8:50/13:50 

[Moderator  should  remind  the  participants  to  communicate  critical  incidents  in  such  a  way 
that  they  are  unclassified.] 

First  Question  Block 

Part  1:  Language  Usage  &  Capability  Requirements: 

Ql:  “Tell  us  about  a  situation  where  having  language  skills  and  capabilities  contributed 
to  mission  success?” 

•  What  was  the  proficiency  level?  How  many  on  the  team  had  language 
capability? 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  Was  the  mission  inside  or  outside  your  AOR? 

•  Did  you  use  an  interpreter?  (Was  the  interpreter  a  contractor  or  military 
linguist?) 

•  Did  you  do  pre-mission  language  prep  ? 

•  Did  you  use  job  aids  (e.g.,  Kwikpoint)  to  help  prepare?  If  yes,  what’s  your 
evaluation  of  the  job  aid? 

Q2:  “Tell  us  about  a  situation  where  not  having  sufficient  language  skill  or  capabilities 
was  a  challenge  for  the  mission  or  degraded  the  mission  outcome?” 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  How  did  it  take  away  from  mission  accomplishment? 

•  How  did  you  overcome  not  having  sufficient  language  capabilities? 

•  How  would  having  better  language  skills  enhance  the  mission? 

Q3:  “Which  SOF  mission  activities  require  language  capabilities?  Please  be  specific  in 
terms  of  language  skill  type  (reading,  listening,  speaking,  writing),  activity  (e.g.,  training 
troops  on  weapon  maintenance,  negotiation,  etc.),  and  criticality  of  the  activity.  Be  as 
descriptive  as  possible  when  talking  about  the  specific  activities.  ” 

Q4:  “What  types  of  language  capabilities  should  the  team  have?  Should  everyone  on  the 
team  (or  only  a  few)  have  some  capability?  What  levels  of  capability  should  the  team 
have?  Should  there  be  multiple  languages  on  a  team?” 

Part  2:  Cultural  Knowledge  and  Awareness  Requirements: 

Q5:  “ Tell  us  about  a  situation  where  having  cultural  knowledge  and  awareness 
contributed  to  mission  success?”  (Provide  as  much  detail  as  possible) 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  Did  you  have  any  pre-mission  culture  training? 

Q6:  “Tell  us  about  a  situation  where  not  having  sufficient  cultural  knowledge  and 
awareness  was  a  challenge  for  the  mission  or  degraded  the  mission  outcome?” 

•  Which  of  the  core  SOF  tasks  (e.g.,  DA,  SR,  FID)  were  involved? 

•  How  did  it  take  away  from  mission  accomplishment? 

•  How  did  you  overcome  not  having  sufficient  cultural  knowledge  and 
awareness? 

•  How  would  having  better  cultural  knowledge  and  awareness  enhance  the 
mission  ? 
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[Moderator  summarizes  first  block  and  reminds  participants  to  add  any  additional 
comments  from  the  first  block  on  the  survey.] 

9:35/14:35 

Second  Question  Block  (Training): 

[Moderator  may  not  get  to  all  of  the  following  questions,  so  they  should  be  mixed  up  across 
groups.  But,  get  to  as  many  as  possible.] 

Q7:  “Where  do  you  think  language  and  culture  training  should  be  provided/offered  in  the 
development  of  a  SOF  operator?  Please  think  about  this  question  across  the  entire  career 
lifecycle  and  in  terms  of  all  forms  of  training  (e.g.,  pre-deployment ).  ” 

•  Should  institutional,  involuntary  training  be  provided  for  [their  SOF 
component]  ? 

Q8:  “We  are  interested  in  your  experiences  with  language/culture  training.  Tell  us  about 
the  most  effective  language/culture  training  in  which  you  have  participated.  Be  sure  to 
consider  all  forms  of  training  (e.g.,  pre-deployment).  Be  descriptive  in  terms  of  why  it 
was  effective.  ” 

Q9:  “Tell  us  about  the  least  effective  language/culture  training  in  which  you  have 
participated.  Be  sure  to  consider  all  forms  of  training  from  initial  acquisition  to 
sustainment  and  enhancement.  Be  descriptive  in  terms  of  why  it  was  not  effective.  ” 

Q10:  “What  are  your  thoughts  on  FLPB  (Foreign  Language  Proficiency  Bonus)?  Are 
you  motivated  to  increase  your  language  proficiency  level  as  a  result  of  this  incentive?” 

QH:  “What  are  your  thoughts  on  language  testing?  Are  the  current  tests  an  adequate 
indicator  of  your  language  proficiency?” 

[Moderator  summarizes  second  block  and  reminds  participants  to  add  any  additional 
comments  from  the  second  block  on  the  survey.] 

10:20/15:20 

[Moderator  will  summarize  entire  focus  group  session.  Be  specific  about  any  themes  that  emerged 
from  the  discussion.] 

Before  we  conclude  does  any  one  have  any  additional  comments  or  topics  related  to  language  or  culture 
that  they  would  like  to  discuss.  [Remind  participants  to  take  a  few  minutes  and  complete  the  survey.] 

[To  SOFLO  Liaison]  Do  you  have  any  thing  you  would  like  to  add? 

Thank  you  all  for  your  generosity  in  taking  the  time  out  of  your  busy  schedules  to  talk  with  us  today. 

[Moderator  ends  interview.] 
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APPENDIX  F:  LCNA  SURVEY  TOPIC  AREAS 


Survey  Topic 

Operator  Version 

Leader  Version 

SOF  Unit  Assignment 

This  section  contained  questions  identifying  respondents’  current  assignment  in  the  SOF  community  (e.g., 
USSOCOM,  USASOC,  AFSOC,  TSOC,  etc.).  Both  operators  and  leaders  received  the  same  set  of 
questions. 

•  Follow-up  questions  were  customized  depending  on  answers  to  the  current  assignment  item. 

•  SOF  operators,  SOF  operators  assigned  to  another  duty,  and  those  currently  in  the  training  pipeline 
were  asked  to  report  their  MOS  or  to  choose  their  SOF  type. 

Language  Background 

This  section  contained  questions  about 
respondents’  language  background,  including: 

•  Native/first  language 

•  Yes/No  in  a  language  coded  position 

•  If  yes,  indicate  current  official  or  required 
language  and  any  additional  languages 

•  Self-rate  current  proficiency 

•  Usefulness  of  current  official  or  required 
language  for  missions 

Leaders  responded  to  the  same  items  as  operators, 
plus  additional  questions,  including: 

•  Current  language  testing  status 

•  Questions  about  receiving  FLPB 

Use  of  Language  and 
Culture  Branching  Items 

These  questions  asked  operators  to  classify  their  dep 
branching  to  inside  or  outside  AOR  sections.  Leader? 
unit’s  deployments  during  their  tenure. 

oyment  experience  in  the  past  4  years  for  use  in 
i  responded  to  the  same  questions,  but  classified  their 
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Survey  Topic 

Operator  Version 

Leader  Version 

Inside  AOR  Use  of 
Language  and  Culture 

Operators  with  relevant  experience  were  asked 
about  the  use  of  language  and  culture  on  their  most 
recent  inside  AOR  deployment.  This  section 
included  questions  on  the  following  topics: 

•  General  questions  about  the  mission 

•  Personal  and  team  language  proficiency 

•  Frequency  and  importance  of  17  language 
tasks/activities  in  relation  to  the  mission 

•  Ease/difficulty  in  meeting  language/culture 
requirements  of  deployment 

•  Types  of  vocabulary  and  modalities  used 

•  Use  of  interpreters  and  job  aids 

•  Preparedness  and  importance  of  language 
and  culture 

•  Pre-deployment  language  and  culture 
training 

Leaders  responded  to  the  same  questions  as 
operators,  but  responded  based  on  their  unit’s 
typical/average  deployment  inside  the  AOR. 

•  Several  questions  referencing  specific 
details  about  the  mission  were  omitted  here 
(e.g.,  details  about  language  proficiency  on 
the  mission,  specific  use  of  interpreters  on 
the  mission). 

•  Leaders  only  responded  to  this  topic  if  they 
indicated  they  were  in  a  position  to  comment 
on  it. 

Outside  AOR  Use  of 
Language  and  Culture 

These  questions  were  identical  to  the  inside  AOR 
questions  with  a  few  exceptions.  Operators  were 
not  asked  about  the  type  of  vocabulary  or  language 
modalities  used.  The  following  were  additional 
topics: 

•  Use  of  language  in  the  deployment  location 

•  Time  spent  maintaining  current  AOR 
language 

Leaders  responded  to  the  same  questions  as  the 
operators,  with  the  following  differences: 

•  Several  questions  referencing  specific 
details  about  the  mission  were  omitted  here 
(e.g.,  details  about  language  proficiency  on 
the  mission,  specific  use  of  interpreters  on 
the  mission). 

•  Leaders  only  responded  to  this  topic  if  they 
indicated  they  were  in  a  position  to  comment 
on  it. 
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Survey  Topic 

Operator  Version 

Leader  Version 

Use  of  Interpreters 

This  section  asked  operators  about  their  general  use 
of  interpreters  on  a  mission  in  the  past  4  years. 

Topic  areas  included: 

•  Pre-deployment  training  on  using 
interpreters 

•  Frequency  of  using  interpreters 

•  Issues  with  using  interpreters 

•  Mission  effectiveness  without  using 
interpreters 

•  Tasks  requiring  the  use  of  an  interpreter 

The  final  question  in  this  section  asked  operators  to 
indicate  which  type(s)  of  interpreters  they  used  in 
order  to  branch  to  the  use  of  interpreter  sections 
below. 

Leaders  were  asked  similar  questions  about  their 
unit’s  use  of  interpreters.  However,  many  of  the 
questions  specifically  asked  about  the  use  of 
interpreters  on  inside  versus  outside  AOR 
deployments  instead  of  across  all  deployments. 

Use  of  Interpreters  (CAT 

I) 

This  section  asked  operators  general  questions  about  their  use  and  experiences  with  using  CAT  I 
interpreters.  Leaders  were  asked  the  same  questions  about  their  unit’s  use  of  CAT  I  interpreters. 

Use  of  Interpreters  (CAT 

II/III) 

This  section  asked  respondents  general  questions  about  their  use  and  experiences  with  using  CAT  II/III 
interpreters.  Leaders  were  asked  the  same  questions  about  their  unit’s  use  of  CAT  II/III  interpreters. 

Use  of  Interpreters  (09L) 

This  section  asked  operators  to  rate  the  overall  effectiveness  and  satisfaction  with  using  an  09L.  Leaders 
were  asked  the  same  questions  from  the  perspective  of  their  unit. 

Individual  Language 
Proficiency  Requirements 

These  questions  asked  operators  to  rate  the  proficiency  needed  (in  each  modality)  for  a  person  in  their  role 
to  be  prepared  for  the  typical  mission.  Leaders  were  asked  to  rate  the  proficiency  needed  for  an  operator  in 
their  unit  to  be  prepared. 

Team  Composition 

This  section  asked  questions  about  the  composition  of  language  proficiency  required  at  minimum  for  a 
team/platoon  member  to  be  prepared  for  the  typical  mission.  Operators  received  a  customized  item  based 
on  the  total  number  of  members  on  their-  team/platoon.  Leaders  received  identical  items  to  answer  about 
the  typical  operator  in  their-  unit. 

•  Units  whose  number  of  team  members  was  unknown  were  asked  to  indicate  the  number  of 
members  on  their  team,  and  then  answered  the  items  about  minimum  proficiency. 

•  Questions  also  asked  about  several  issues  and  their  importance  to  team  composition. 
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Survey  Topic 

Operator  Version 

Leader  Version 

Reaction  to  Admiral 
Olson’s  Memo 

Both  operators  and  leaders  were  asked  if  they  were  aware  of  Admiral  Olson’s  memo  and  to  identify  any 
challenges  in  achieving  the  goals  indicated.  They  were  also  asked  about  what  could  be  done  to  facilitate 
achieving  these  goals. 

Language  and  Culture 
Training 

This  section  included  two  branching  questions  to 
identify  the  most  recent  type  of  government 
sponsored  training  received. 

Initial  Acquisition 

Training  (IAT)  Evaluation 

This  section  asked  questions  about  the  IAT 
operators  received.  Topics  included: 

•  Source  and  mode  of  training 

•  Curriculum  evaluation 

•  Instructor  evaluation 

•  Usefulness  of  training 

The  leader  version  of  these  items  asked  about 
operators’  language  proficiency  when  arriving  at 
the  unit. 

Sustainment/Enhancement 
Training  (SET) 

Evaluation 

This  section  asked  questions  about  SET  similar  to 
the  IAT  questions.  Additionally,  questions  about 
changes  in  proficiency  levels  after  training  were 
asked. 

Only  CLPMs  were  asked  about  SET,  see  CLPM 
Training  Items. 

Immersion 

Operators  were  asked  general  questions  about  the 
most  recent  type  of  government  sponsored 
immersion  training  they  participated  in.  Topics 
included: 

•  Duration  and  location  of  training 

•  Effectiveness  and  usefulness  of  training 

•  Changes  in  proficiency  as  a  result  of 
training 

Leaders  were  asked  about  the  types  of  immersion 
opportunities  available  at  their-  unit  and  then 
branched  based  on  their  response.  Leaders  could 
have  received  general  immersion  training  questions 
on: 

•  CONUS  Immersion  Training 

•  OCONUS  Immersion  Training 

•  Both  CONUS  and  OCONUS 

Leaders  whose  units  do  not  provide  immersion 
opportunities  were  asked  a  general  question  about 
recommendations  for  an  immersion  training 
program. 

Culture  Training 

Operators  were  asked  general  questions  about  the  most  recent  government  sponsored  culture  training  they 
received.  Leaders  responded  to  the  same  questions  about  the  culture  training  their  operators  receive. 
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Operator  Version 

Leader  Version 

Self-Study  Materials 

This  section  asked  operators  questions  about  their 
language  facility  and  resources.  Topics  included: 

•  Types  of  resources  available 

•  Effectiveness  of  available  resources 

•  Ease/difficulty  in  accessing  resources 

•  Time  spent  on  language 
learning/maintenance 

See  CLPM  Training  Items. 

Attitudes  towards 

Language 

These  questions  assessed  respondents’  interest  and 
motivation  for  learning  a  foreign  language. 
Respondents  were  also  asked  to  indicate  their 
confidence  in  their  ability  to  perform  12 
tasks/activities  using  their-  language  skills. 

Training  Emphasis 

Operators  and  leaders  were  asked  to  indicate  the  amount  of  training  emphasis  necessary  for  a  number  of 
culture  and  language  tasks/activities  to  prepare  operators  for  their-  missions.  Training  areas  included: 

•  Giving  commands 

•  Using  street  dialect 

•  Reading  signs,  graffiti,  and  maps 

•  Rules  of  etiquette  in  the  deployment  region 

•  Customs,  conventions,  and  norms  for  behavior 

Testing  Background 

Operators  responded  to  general  questions  about  their  testing  background,  including  DLPT,  OPI,  and 

DLAB.  Leaders  responded  to  the  same  questions  about  the  testing  background  of  their  operators.  Topics 
included: 

•  General  questions  about  the  most  recent  DLPT  and/or  OPI  taken 

•  Issues  when  taking  the  DLPT  or  OPI 

•  How  related  the  DLPT  or  OPI  is  to  the  job 

•  Accuracy  of  the  DLAB  in  assessing  language  learning  aptitude 

Standard  Change 

Operators  and  leaders  were  asked  two  questions  about  their  reaction  to  the  recent  test  of  record  change  for 
USSOCOM. 
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Monetary  and  Non¬ 
monetary  Incentives 

In  this  section,  operators  and  leaders  were  asked  their  opinion  on  making  language  part  of  the  promotion 
process.  Additionally,  they  indicated  how  motivated  operators  are  to  acquire/maintain  proficiency  based 
on  several  incentives.  Incentives  included: 

•  Mission  success 

•  Monetary  incentives  (i.e.,  FLPB) 

•  Opportunities  for  training 

•  Immersion  opportunities 

Foreign  Language 
Proficiency  Bonus 
(FLPB)-specific  Items 

This  section  asked  operators  about  their  experiences  with  FLPB.  Leaders  were  asked  about  their-  units’ 
experiences.  Questions  about  FLPB  included: 

•  How  motivating  is  FLPB 

•  How  fair  are  the  procedures  related  to  FLPB 

•  How  could  FLPB  be  more  motivating 

Barriers 

Questions  in  this  section  asked  operators  and  leaders  about  the  organizational  climate  and  support  at  their 
unit.  Topics  included: 

•  Grading  your  chain  of  command 

•  Potential  barriers  to  acquiring/maintaining  proficiency 

•  Language  priority  when  compared  to  other  training  requirements 

Unit  Specific  Plans 

If  respondents  wanted  to  provide  additional  comments  or  recommendations,  this  section  included  an  open- 
ended  question  regarding  improving  language  capability  in  the  unit.  Leaders  were  asked  what  they  are 
currently  doing  at  their  units  to  improve  capability. 

Demographics 

This  section  contained  background  questions  about 
survey  participants.  Items  included: 

•  Tenure 

•  Deployment  in  the  past  2  years 

•  Number  of  deployments  inside  v.  outside 
the  AOR 

•  Grade 

Leaders  were  asked  similar  questions,  including: 

•  Position 

•  Staff  section 

•  Level  of  command 

•  Grade 

•  Tenure 
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Command  Language 
Program  (CLP)  Training 

Leaders  were  asked  general  questions  about  the 

CLP  at  their  unit.  Topics  included: 

•  Type  of  training  available 

•  Overall  quality  of  various  aspects  of  training 

•  Satisfaction  with  quality  of  CLP 

Leaders  Only 

This  section  only  asked  leaders  questions  about 
training  issues,  mission  planning,  and  resource 
availability. 

Special  Operations  Forces 
Culture  and  Language 
Office  (SOFCLO) 

Support 

Leaders  were  asked  questions  about  the  SOFCLO, 
including: 

•  Awareness  of  the  SOFCLO 

•  Support  received  from  the  SOFCLO 

Command  Language 
Program  Manager 
(CLPM)  Training  Items 

This  section  only  asked  CLPMs  questions  about  the 
language  facility  and  training  at  their  unit.  Topics 
included: 

•  Curriculum  evaluation 

•  Instructor  evaluation 

•  Student  evaluation 

•  Resource  availability 

•  SET 

Instructor 

These  questions  were  specific  to  instructors  only. 
Topics  included: 

•  Student  evaluation 

•  Incentives  that  motivate  students 

•  Barriers  preventing  students  from 
acquiring/maintaining  proficiency 

2/25/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011002] 


Page  43 


SOF  Language  and  Culture  Needs  Assessment  Project 


Methodology  Report 


APPENDIX  G:  BRANCHING  CHART 


SOF  Unit  Assignment 
Language  Background 
Use  of  Language  on  Deployment 
Use  of  Interpreters  (General) 
Individual  Proficiency  Requirements 
Team  Proficiency  Requirements 
Admiral  Olson's  Memo 
Language  Training  -  IAT  and  SET 
Immersion  Training 
Culture  Training 
Language  Facility  and  Resources 
Attitudes  toward  Language 
Training  Emphasis 
Language  Testing 
Standard  Change 
Incentives 
FLPB 

Organizational  Climate  and  Support 
Unit  Specific  Plans 
Demographics 


SOF  Unit  Assignment 
Language  Background 

Use  of  Interpreters  (General) 

Admiral  Olson's  Memo 
Language  Training  -  IAT  and  SET 
Immersion  Training 
Culture  Training 
Language  Facility  and  Resources 
Attitudes  toward  Language 
Training  Emphasis 
Language  Testing 
Standard  Change 
Incentives 
FLPB 

Demographics 


SOF  Unit  Assignment 
Language  Background 
Use  of  Language  on  Deployment 
Use  of  Interpreters  (General) 
Individual  Proficiency  Requirements 

Admiral  Olson's  Memo 
Language  Training  -  IAT  and  SET 
Immersion  Training 
Culture  Training 
Language  Facility  and  Resources 
Attitudes  toward  Language 
Training  Emphasis 
Language  Testing 
Standard  Change 
Incentives 
FLPB 

Organizational  Climate  and  Support 
Unit  Specific  Plans 
Demographics 


Language  Background 
Use  of  Language  on  Deployment 
Use  of  Interpreters  (General) 

Admiral  Olson's  Memo 


Language  Testing 


Demographics 


J  K 


J 


V. 


J  ^ > 


2/25/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011002] 


Page  44 


SOF  Language  and  Culture  Needs  Assessment  Project 


Methodology  Report 


2/25/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011002] 


Page  45 


SOF  Language  and  Culture  Needs  Assessment  Project 


Methodology  Report 


Leadership 

SOF 

r~ 


r  > 

SOF  Unit  Leader 

^ _ / 

— CLPM 

—Language  Office 
Personnel 

v _ / 

r  a 

Instructor 

v _ > 

r  a 

Other 

Civilian/Military 

Personnel 

v _ y 

f  \ 

f  \ 

SOF  Unit  Assignment 

SOF  Unit  Assignment 

SOF  Unit  Assignment 

SOF  Unit  Assignment 

Demographics 

Demographics 

Demographics 

Language  Background 

Language  Background 

Language  Background 

Language  Background 

Use  of  Language  on  Deployment 

Use  of  Language  on  Deployment 

Use  of  Interpreters  (General) 

Use  of  Interpreters  (General) 

Use  of  Interpreters  (General) 

Individual  Proficiency  Requirements 

Individual  Proficiency  Requirements 

Individual  Proficiency  Requirements 

Individual  Proficiency  Requirements 

Team  Proficiency  Requirements 

Team  Proficiency  Requirements 

Admiral  Olson's  Memo 

Admiral  Olson's  Memo 

Language  Training  -  IAT  and  CLP 

Language  Training  -  IAT  and  CLP 

Immersion  Training 

Immersion  Training 

Culture  Training 

Culture  Training 

Leader-Specific  Items 

CLPM  Specific  Items  [CLPM  ONLY} 

Instructor  Specific  Training  Items 

Training  Emphasis 

Training  Emphasis 

Training  Emphasis 

Language  Testing 

Language  Testing 

Language  Testing 

Language  Testing 

Standard  Change 

Standard  Change 

Standard  Change 

Standard  Change 

Incentives 

Incentives 

FLPB 

FLPB 

Organizational  Climate  and  Support 

Organizational  Climate  and  Support 

Unit  Specific  Plans 

Unit  Specific  Plans 

SOFCLO  Support 

SOFCLO  Support 

2/25/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011002] 


Page  46 


SOF  Language  and  Culture  Needs  Assessment  Project 


Methodology  Report 


2/25/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011002] 


Page  47 


SOF  Language  and  Culture  Needs  Assessment  Project 


Methodology  Report 


APPENDIX  H:  ADMIRAL  OLSON’S  MEMO 


UNITED  STATES  SPECIAL  OPERATIONS  COMMAND 

OFFICE  OF  THE  COMMANDER 

7701  TAMPA  POINT  BLVD. 

MACDILL  AIR  FORCE  BASE,  FLORIDA  33621-5323 


30  October  2009 


MEMORANDUM  FOR  SEE  DISTRIBUTION 

SUBJECT:  Participation  in  United  States  Special  Operations  Command  Language 
and  Culture  Needs  Assessment  Survey 


1 .  Request  your  support  of  and  participation  in  the  United  States  Special  Operations 
Command  Language  and  Culture  Needs  Assessment  (LCNA).  This  survey  provides 
information  to  leaders,  planners,  and  trainers  to  ensure  Special  Operations  Forces 
personnel  have  the  language  and  culture  capabilities  needed  to  support  their 
missions. 

2.  Our  language  and  culture  staff  conducted  a  series  of  focus  groups  throughout  the 
Command.  The  results  were  used  to  develop  survey  questions.  The  LCNA  will  be 
administered  online  from  2  through  13  November  2009. 

3.  The  survey  link  will  be  on  Defense  Knowledge  Online  and  sent  in  emails  to 
component  language  managers  for  distribution.  The  survey  link  is  below. 
https://swa.qualtrics.co  m/SE?SID=SV_3dZAg1ijsxpYXvC&SVID=Prod 

4.  The  point  of  contact  is  Mr.  Donnelly,  DSN  299-6040,  commercial  (813)  826-6040, 
or  emailjohn.donnelly@socom.mil  orjohn.donnelly@hq.socom.smil.mil. 


Admiral,  U.S.  Navy 
Commander 


DISTRIBUTION: 

COMMANDER,  UNITED  STATES  ARMY  SPECIAL  OPERATIONS  COMMAND,  2929 
DESERT  STORM  DRIVE,  FORT  BRAGG,  NC  28310-9110 

COMMANDER,  MARINE  CORPS  FORCES  SPECIAL  OPERATIONS  COMMAND, 
PSC  BOX  20116,  CAMP  LEJEUNE,  NC  28542-0116 

COMMANDER,  NAVAL  SPECIAL  WARFARE  COMMAND,  2000  TRIDENT  WAY, 
SAN  DIEGO,  CA  92155-5599 
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socc 

SUBJECT:  Participation  in  United  States  Special  Operations  Command  Language 
and  Culture  Needs  Assessment  Survey 


COMMANDER,  AIR  FORCE  SPECIAL  OPERATIONS  COMMAND,  100  BARTLEY 
STREET,  CMD  STE,  HURLBURT  FIELD,  FL  32544-5273 

COMMANDER,  JOINT  SPECIAL  OPERATIONS  COMMAND,  P.O.  BOX  70239, 
FORT  BRAGG,  NC  28307-5000 

PRESIDENT,  JOINT  SPECIAL  OPERATIONS  UNIVERSITY,  357  TULLY  STREET, 
BLDG  90503,  HURLBURT  FIELD,  FL  32544-5278 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  JOINT  FORCES 
COMMAND,  1721  PIERSEY  STREET,  NORFOLK,  VA  23511-2490 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  AFRICA,  UNIT  30401 , 
APO  AE  09107 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  CENTRAL,  7715 
BOUNDARY  BLVD,  MACDILL  AFB,  FL  33621-5101 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  EUROPE,  UNIT  30400 
APO  AE  09128-4209 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  KOREA,  UNIT  15622, 
APOAP  96205-0328 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  PACIFIC,  BOX  64046, 
CAMP  H.M.  SMITH,  HI  96861-4046 

HEADQUARTERS  SPECIAL  OPERATIONS  COMMAND  -  SOUTH,  29350  CORAL 
SEA  BLVD,  HOMESTEAD  AIR  RESERVE  BASE,  FL  33039 
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